IBM Research Progresses Field of Human-Computer Interaction (HCI) IBM Research Blog

Share this publish:

The ACM CHI Conference on Human Factors in Computing Systems may be the premier venue for publishing searching in human-computer interaction (HCI). This season, CHI was cancelled because of COVID-19, even though some workshops is going to be held virtually. We intend to make recorded video presentations from IBM authors obtainable in this publish when available. Some IBM authors produced video presentations their papers. You’ll find links to those videos within the listing of recognized IBM works in the finish of the publish. SIGCHI may also make recorded presentations for full papers obtainable in the ACM Digital Library. IBMResearch were built with a strong body of labor recognized to CHI 2020, including nine full papers, six late-breakingwork papers, three courses, one demo, six co-organized workshops, a panel,and a unique Interest Groups (SIG) meeting. Included in this, one paperreceived the prestigiousCHI Best Paper award,andanother paper received anHonorable Mention award. These awardsrepresent the HCI community’s recognition to find the best-quality searching.

IBM Research’scontributions to CHI 2020 focus oncreating and designing AI technologies that focus on user needs and societal values, spanning the themes of novel human-AI partnerships, AI UX and style,trusted AI,and AI for ease of access.In this publish, wehighlighta choice of our focus on these topics, and supply a complete listing of recognized IBM work on the finish.

Novel Human-AI Partnerships

As AI technologies become more and more prevalent and capable, researchers at IBM are envisioning novel types of human-AI partnershipsand investigating a persons needs around them.

In apaperthatreceivedaBest Paperaward, IBM researchers createdahuman-AI cooperative word guessing game and studied how people form mental types of their AI game partner. Through two studies—a think-aloud study and alarge-scaleonlinestudy—researchers investigated how peoplecome to know the way the AI agent operates.

Hanging around, the agent gives clues in regards to a target wordandparticipants must guess the prospective word based onthoseclues.The paper investigates what conceptual types of AI agentsshouldinclude and describesthree componentsvitalforpeople to create an accuratemental model ofanAI system:local behavior,which includes conceptions of what types of hints the AI agent will probably give or respond better to,global behavior,which includes conceptions of methods the AI agent has a tendency to take part in the game,andunderstanding distribution,which includes conceptions for example set up AI agent is aware of specific people or attributes.The paper offers these groups as aguideforconceptual model development of all types of AI systems.In a sizable-scale study, they discovered that those who have better estimates from the AI agent’s abilities are more inclined to win the sport, and those that shed more pounds frequently have a tendency to over-estimate the AI agent’s abilities.


Figure 1. Screenshot from the human-AI cooperative word guessing game

IBM Researchers, alongwith our academic partners within the IBM AI Horizons Network,have createda human-multi-agentnegotiation platform in whichpeoplehaggle with existence-sized humanoid avatars inside a simulated street market.Theseavatars can sense when they’re being addressed by observing the customer’s mind orientation, and they could undercut each other’s offers. The work breaks new ground in a number of respects, including direct settlement between humans andAIagents via natural language,theuse of non-verbal human-agent communication via mind orientation,andmulti-lateral negotiationsituatedinan immersive atmosphere.Alate-breakingworkpaperdescribes this platform andassociated research challenges, and early experimentsindicatethat human participants enjoyedthe experience.

Thisnegotiation platform supplies a grounds for a brand new worldwide AI competition known as HUMAINE 2020(Human Multi-agent Immersive Settlement), whichis slated to become held along with IJCAI 2020. Each one is thanks for visiting join your competition!View the net sitefor details andtosignup, andwatch asneak previewof the immersive atmosphere where the competition is going to be held.


Figure 2. Prototype of human-multi-agent settlement platform

AI Consumer Experience (AIUX) and Design

HCI research frequently strives to create design guidelines and practical approaches thathelppractitionersmake technology more user-friendly. The age of AI requires innovation in design methods and procedures that concentrate on the initial challengesinherent touser interactions with AI.

Oneof these challengesisinexplainability. Understanding AI is really a universal needforpeople who develop, use, manage, regulate, or may take a hit byAI systems. However, AI technology is complexandoftendifficult to understand, like the high-performing, yetopaque,ML modelsbuilt withdeep neural systems. To tackle this issue,ML scientific study has developed aplethora of strategies to generate human-consumable explanations for ML models. At IBM, our open-source toolkitAI Fairness 360makesthese techniques easily accessibleto ML developers.

In apaper that receivedanHonorable Mentionaward, IBMresearchers studied the practices of twenty design professionals focusing on sixteendifferentAI products to know the look space of explainable AI and discomfort points in current design practices. Thisresearch created some actionable understanding that bridges user needswithexplainable AI (XAI) techniques, includingtheXAI Question Bank, a summary of prototypical questions usershavewhen askingfor explanations in AI systems, and style guidelines to deal with these user questions.

Researchers at IBM also positively practice innovative,user-centered design approaches within their work developing new AI systems. In acase study paper, IBM researchers appliedstorytelling techniques to design and validate theUXof an AI assistant for theoil &gas industry.Inspired bytheirfindings fromextensive user research,the researchers produced a sketch-based videodetailing the consumer experienceof theAI assistantand howitwould reshape and empower understanding workers’ everyday tasks.This workshowsthe advantages of this sketch-basedapproachcomparedto conventional methods for validating UX design,such aswireframes, mock-ups, and storyboards.

In alate-breaking-work paper, IBM researchers documentedhow humanitarian aid workers could deal with an AIsystemto gain insights from various data sources to aid decision-making practices round the allocation of sources to assist intentionally displaced peoples. Within this collaboration between IBM Research and also the Danish Refugee Council (DRC), the teamcombinedempirical datawitha scenario-based design approachtoderive current and future scenariosdescribingcollaboration betweenhumanitarian aid experts and AIsystems.

Reliable AI

IBM Scientific studies are positively building and enabling AI solutions people can trust. HCI researchers play a huge role such efforts, as reflected in 2 late-breaking-work papers.

The amount of AI modelsbeing usedin high-stakes areas such asfinancial risk assessment, medical diagnosis,andhiring keeps growing. Correspondingly, the requirement for elevated transparency and rely upon such models becomes evenmore relevant. Although thespecificsofthesemodels differacross domains, allofthese models face exactly the same challenge: how tocollectand disseminate themost information in regards to a model — it’sfacts—in a means thatisaccessible to developers,data scientists, along with other business stakeholders who decide about this model’s use.To address this needfor transparency in how machine learning models are produced, IBM researchers havepromoted the idea of a FactSheet, an accumulation of details about an AImodel or service that iscaptured through the machine learning lifecycle.Through user research with data scientists and AI developers, IBM researchers developed some recommendations which help guide the introduction of FactSheets,such as supplying user guidance for authoring details , and the way to report the details towards the various involved stakeholders.These recommendationsalso provideAI system builders having a greaterunderstanding from the underlying HCI challenges associated with documentation AI systems.


Figure 3. Prototype of FactSheet

Information is the building blocks of derived understanding and intelligent technologies. However, biases in data, for example gender and racial stereotypes, could be propagated through intelligent systems and amplifiedinend-user applications. IBM researchersdevelopeda general methodology to evaluate the amount of biases inside a datasetby calculating the main difference of their data distribution having a reference dataset usinga metric calledMaximum Mean Discrepancy. Evaluation results reveal that thismethodology might help domain experts to discover various kinds of data biases used.

AI for Ease of access

IBM Scientific studies are dedicated to creatinginclusive and accessible technologies, andourefforts continue intheAI era. Ease of access researchers at IBM, along with our academic partners in US and Japan,publishedtwo full papers and 2 late-breaking papers at CHI 2020.The team continues to be exploring methods to help visually impaired individuals to recognize real-world and enhance the caliber of existence. The work they do leveragesrapidly-maturingAI technologiesthat arecapable of understandingthesurrounding environmentof a visually-impaired person, for example products inside a shop or empty seats on the train. They developed multiple systemsthatenable visually-impaired individuals to become more independent on a holiday, in an office, or athome:a system using wearable cameras, asmartphone-based personal object recognition system known as ReCog, which may be trained by blind users, a robotic navigation system for dynamic targetsthat helps peoplefind and navigate to things inside a room, such asan empty chair, along with a smartphone-based navigation systemthathelps blind people stand it lines.


Figure 4. Interfaces of ReCog

Additionally to the contributions towards the CHI literature, IBM investigator Dr. Shari Trewin offered being an Ease of access Co-Chair. She involved in an unparalleled effort to createaccessible paper proceedings. Her work helps to ensure that every paper printed at CHI will come in an accessible format for those people from the CHI community.

Recognized Papers

Recognized Late-breaking Work

Recognized Demo

  • Josh Andres.Neo-Noumena

Recognized Courses

  • Q. Vera Liao, Monider Singh,Yunfeng Zhang, Rachel K.E. Bellamy.Introduction to Explainable AI.
  • Yunfeng Zhang, Rachel K.E. Bellamy, Monider Singh, Q. Vera Liao.Introduction to AI Fairness
  • Josh Andres.Inbodied Interaction 102: Comprehending the Selection and Use of Non-invasive Neuro-physio Measurements for Inbodied Interaction Design

Recognized Workshops

Recognized Panel

  • Dakuo Wang.From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That May Deal with People

AcceptedSpecial Interest GroupsMeeting

  • Michael Muller.Queer in HCI: Supporting LGBTQIA+ Researchers and Research Across Domains.Virtual meeting

Recognized Workshop Papers

More AI tales

In the annual Conference on Empirical Methodsin Natural Language Processing (EMNLP), IBM Research AI is presenting 30 papers within the primary conference and 12 findings that together try to advance the concept of natural language processing (NLP). Of note, IBM Research AI printed the 2009 yeara new, four-part mastering language taxonomy that people believe will [&hellip]

Recording and structuring common understanding in the real life to really make it open to personal computers is among the foundational concepts of IBM Research. The actual-world details are frequently naturally organized as graphs (e.g., internet, social systems) where understanding is symbolized not just by the information content of every node, but additionally [&hellip]

Launched in 2018, the Rensselaer-IBM Artificial Intelligence Research Collaboration (AIRC) is really a multi-year, multi-billion dollar partnership boasting a large number of ongoing projects in 2020-2021 involving greater than 80 IBM and RPI researchers trying to advance AI.