Trust, but verify: The quest to measure our trust in AI

Artificial intelligence is a growing presence in our lives and has great potential for good — but people working with these systems may have low trust in them. One research team at the Center for Accelerating Operational Efficiency is working to address that concern by testing a U.S. Department of Homeland Security-funded tool that could help government and industry identify and develop trustworthy AI technology.

“Things that would lead to low trust are if the technology’s purpose, its process or its performance were not aligned with your expectations,” said CAOE researcher Erin Chiou, who is also an associate professor of human systems engineering in The Polytechnic School, part of the Ira A. Fulton Schools of Engineering.

The tool, called the Multisource AI Scorecard Table (MAST), is aimed at aligning those expectations. It’s based on a set of standards originally developed to evaluate the trustworthiness of human-written intelligence reports. It uses a set of nine criteria, including describing the credibility of sources, communicating uncertainties, making logical arguments and giving accurate assessments.

To test whether MAST can effectively measure the trustworthiness of AI systems, volunteer groups of transportation security officers interacted with one of two simulated AI systems that the ASU team created.

One version of the simulated AI system was built to rate highly on the MAST criteria. A second version was built to rate low on the MAST criteria. After completing their tasks with one of these systems, the officers completed a survey that included questions on how trustworthy they consider AI.

The Transportation Security Administration’s federal security directors from Phoenix Sky Harbor International Airport, San Diego International Airport and Las Vegas Harry Reid International Airport each organized volunteer officers to participate in the study.

CAOE has a history of working with the TSA. Previously, Chiou’s team participated in piloting a new AI-powered technology at one of Sky Harbor’s terminals. The new screening machine uses facial recognition to help a human document checker verify whether a person in the security line matches their ID photo. That study showed that the technology increased accuracy in screening. The current project is testing whether new, MAST-informed features will affect officers’ trust perceptions and performance with the technology.

“The Transportation Security Administration team in Arizona has had the privilege to partner with ASU and CAOE for the past several years. We are particularly excited to have the opportunity to partner with them on this project involving artificial intelligence,” said Brian W. Towle, the assistant federal security director of TSAArizona. “With the use of AI rapidly growing across government and private sector organizations around the globe, there is significant value in increasing public awareness and confidence in this technology.”

For the fieldwork phase of the current project, the officers viewed images of people and ID photos as if they were in the document checker position. The simulated AI system gave them recommendations for whether the images and ID photos matched. The officer makes the final decision about whether to let a person through the line, which is why their trust in the AI recommendation matters.

AI-powered face matching technology is already in place in at least one terminal of Sky Harbor, Chiou says, as well as similar systems in airports in the United Kingdom and Canada. While it may take a while for airports nationwide to acquire this technology, we are likely to encounter it more and more in the future as we travel.

If the ASU team is able to show that the MAST tool is useful for assessing AI trustworthiness, it will help in building and buying systems that people can rely on, paving the way for AI’s smooth integration into critical sectors, protecting national security and multiplying its power for positive impact.