How good are machines at reading, compared to humans? What technology is driving breakthroughs in Natural Language Processing (NLP) and reading comprehension, and what are the implications?

Last week provided us with attention-grabbing headlines from Bloomberg, Newsweek, Engadget, Wired and other news outlets announcing that based on the reading comprehension test scores of artificial intelligence (AI) models from Microsoft and Chinese e-commerce giant Alibaba, “robots can now read better than humans” and that this development could potentially put “millions of customer service jobs at risk due to automation”

Both Microsoft and Alibaba tested the language comprehension skills of their respective Artificial Intelligence (AI) algorithms with the Stanford Question Answering Dataset (SQuAD), a reading comprehension test that features more than 100,000 questions on Wikipedia articles which must be answered exactly for success.

Thanks to its powerful NLP capabilities, the AI model from Alibaba’s Institute of Data Science of Technologies slightly outperformed humans with a score of 82.44 to 82.31. The following day, the AI model from Microsoft notched an 82.65, setting the bar even higher. Results of the scores of several different AI models, including those from Microsoft and Alibaba, are available on the Stanford SQuAD website here: https://rajpurkar.github.io/SQuAD-explorer/

After looking at these results more closely, is there sufficient cause for concern at this news? Or is this just premature enthusiasm and nothing more? And if a tidal shift is in fact racing toward us, can we predict a timeline for this monumental change across multiple industries to occur?

According to those on the cutting edge of these technologies, there are a wide range of applications that will reduce the need for human intervention. Alibaba chief scientist Luo Si, whose AI scored so highly in the SQuAD tests, said in a statement that “objective questions such as ‘what causes rain’ can now be answered with high accuracy by machines… The technology underneath can be gradually applied to numerous applications such as customer service, museum tutorials and online responses to medical inquiries from patients, decreasing the need for human input in an unprecedented way.”

But other experts and industry observers are not as concerned. Adrian Lee of Gartner Research offered his comments in reactioin to this news:

“The article did not continue on to reveal which were the millions of jobs at risk. So, the alarmist stance that massive job cuts were on the horizon didn’t worry me. Yet. To put things into perspective, Gartner currently has over 900 research articles on the topic of conversational platforms, with more than 46 different analysts investigating the area of conversational AI.”

“We still need to make sure the semantic aspects of the technology work. This is what drives the ‘understanding’ of human by machine. As chatbots proliferate across customer service, financial services, retail, government and even healthcare; the ability of technology vendors to provide contextually aware, multiturn-based natural-language conversations presents an ongoing challenge to natural language understanding (NLU) handling. This is especially the case in the areas of semantic parsing and natural-language inferences.”

“Ambiguities in semantics arise when there are multiple grammatical interpretations possible. ‘Giant road bike’ and ‘Giant road bike’ on a digital commerce site can mean very different outcomes if the semantics engine cannot infer that ‘Giant’ refers to a brand of bicycle and not the size of the bike in question. But you already knew that.”

For further reading on the fascinating subject of NLP and AI, we recommend the following links to get you started on the journey to learn more:

https://www.cnet.com/news/new-results-show-ai-is-as-good-as-reading-comprehension-as-we-are/

https://technode.com/2018/02/07/iflytek/

https://www.eyefortravel.com/mobile-and-technology/china-voice-tech-rises-over-over-big-four

Thanks for reading today’s Tech Blog! Do you have any thoughts on the potential of technologies behind machine language recognition and natural language processior, or its potential for the future? Feel free to drop us a line via social media or our Contact Us form and let us know – and what you might like to see in future posts!