2 minute read
How Good are AI Reading Comprehension Systems? By Dr Keith Darlington
How good are AI reading comprehension systems?
There are a range of successful AI reading comprehension applications in current use, although they are still a work in progress. In this article, I describe some of the problems associated with reading comprehension applications. I also give brief insights into how they work and describe some benchmark tests in use – some of which show claim performance levels exceeding human capabilities.
BY DR KEITH DARLINGTON
Reading comprehension is something we learn during our early school years. But it is an application area that is very important for AI systems because search engines can improve by delivering better answers if they can comprehend the meaning of user queries. Search engines usually answer queries by showing lists of Websites ranked according to their perceived importance. However, this may be of little benefit to a user who has to sift through long lists only to encounter many irrelevant references. Accessing precise information with its intended meaning is crucial to successful systems. There are many other uses of reading comprehension systems, such as using chatbots, virtual agents, and reading road signs in autonomous cars.
Reading comprehension poses a formidable challenge for the competency of AI systems because they exemplify the chasm between humans and AI: a lack of understanding. This inability for AI systems to understand as humans do is a difference that some say is irrevocable because the machine will never understand semantics and human intentions in the same way humans do.
However, while it may be true that AI systems do not understand the meaning of language as humans do, that does not preclude them from simulating tasks that achieve certain levels of understanding. For example, suppose I said that my friend could run the 100 metres in less than 10 seconds. We would infer that my friend is a good athlete. It may be tempting to believe that the machine would need to have a similar understanding of such life events to draw the same conclusion. But answering this type of question is not beyond the capabilities of AI systems because this type of knowledge describing relationships between attainment and achievement level could be encoded and, therefore, inferences made that reflect some forms of human understanding. Tasks of this kind could be implemented, if a relatively small subset of natural language is used in a specific domain – such as chatbot sales assistants.
HOW AI READING COMPREHENSION SYSTEMS WORK
Most reading comprehension AI systems work by reading queries, comprehending, and providing answers. The user would ask questions about written sections of text in a particular document (or perhaps a search of the World Wide Web) with answers given in a presentable concise format.
