Best of both worlds: research combines power of humans and machines to improve question answering

When you have a question that needs to be answered, where do you turn? A machine, like Google or Siri? Or perhaps it’s a more complicated problem that requires the insight of a human expert. Neither machines nor humans are perfect, but what if you could combine their unique strengths to improve question answering?

That’s the focus of a new study by Ohio State University Computer Science and Engineering Assistant Professor Huan Sun. Her proposal, “Advancing Human and Machine Question Answering via Human-Machine Collaboration” will receive approximately $499,000 in funding from the Army Research Office. The Army Research Office is an element of the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory.

Humans and intelligent machines are the two main channels for question answering (QA). People can now get direct and precise answers from voice assistants, rather than a long list of relevant documents from traditional search engines. Online community platforms such as Quora demonstrate how traditional in-person enquiring is becoming a powerful way of seeking help from a network of online experts.

Despite the rapid development of each channel, they still have their limitations, said Sun.

“Current machine QA systems are good at simple factoid questions, and our benchmark results show that their performance degrades dramatically when questions become more complex, such as involving multiple entities and relations,” she said. “Meanwhile, although human QA systems can answer complex, non-factoid questions, they tend to be much more expensive and time-consuming to use.”

While most existing studies focus on improving one or the other, Sun’s project aims to develop novel human-machine collaboration mechanisms to facilitate both QA channels, with the potential to greatly advance each system.

“The respective limitations of current machine and human QA systems can be largely remedied by the other,” she said. “Humans can easily resolve the ambiguities in complex questions that confuse machines, while machines can provide useful insights by analyzing large amounts of data and make human QA systems more automated.”

Sun’s team will develop innovative techniques that foster synergy between humans and machines. For example, when searching for answers to a user question, an intelligent machine can interact with the human user to resolve question ambiguity, boost answer accuracy, and improve user trust. Moreover, the machine can learn from such interactions to become smarter over time.

“Human-machine interaction is—and will continue to be—a vital part of Army operations, particularly with respect to real-time analysis of information, decision making based on queries from databases, and efficient information extraction,” said Dr. Joseph Myers, mathematical sciences division chief in the Army Research Office. “This research, addressing the challenges faced by human and machine question and answering systems, could play a significant role in improving all of those aspects of the Army's mission.”

“Our grand vision is that humans and machines should team up as an integrated complex system for effective and efficient question answering,” Sun added. “But the techniques we’re proposing can potentially be generalized to many other scenarios where human and machine intelligence need to be combined in an algorithmic manner.”

Sun’s research has focused on advancing both machine intelligent and human collaborative systems with the ultimate goal of improving question answering and easing decision-making. Her work has applications in healthcare, education, business and cybersecurity.

by Meggie Biss, College of Engineering Communications | biss.11@osu.edu