From the Future: Unexpected AI applications

By SPENCER KELLY news Writer

In our inaugural edition, From the Future looked at AI research at n otre d ame related to broad philosophical, political and practical issues. This week, we profile three n otre d ame researchers using AI for unusual or unexpected applications in fields as diverse as peace studies, science and the humanities. These projects demonstrate how AI disruption will be truly ubiquitous, but also how n otre d ame is a leader in conjuring creative ways to use AI to advance scholarship in a multidisciplinary fashion.

r eading into the past with ai vision

Walter s cheirer, d ennis o d oughty c ollegiate Associate Professor, c omputer s cience and e ngineering

As far back as his undergraduate years, Walter s cheirer, d ennis o d oughty c ollegiate associate professor of computer science and engineering, envisioned novel applications for AI. h is vision, as it were, was that computers themselves would be able to see.

“I thought [computer vision] was interesting, because it’s a really broad capability,” s cheirer said. “ m y background is pretty diverse, so I was interested in social science questions and humanistic questions … I wanted to develop increasingly sophisticated algorithms that could help on problems that I thought were really hard and interesting in the world of digital humanities.” s cheirer joined forces with classicists and medievalists from around n otre d ame and got to work on an AI program that could transcribe the ancient documents into text form. The AI was trained using supervised learning, where computer scientists would feed examples of already transcribed texts into their AI so it could learn to “read” Latin. s cheirer noted that training the AI involved interdisciplinary collaboration. c omputer scientists had to do the technical programming, but humanities scholars had to help explain the basics of medieval texts and provide the input data (the sample transcriptions) required for machine learning.

Years later in 2017, a group from r oma Tre University contacted s cheirer about a project that was right up his alley of computer vision in the humanities. The group had been working on transcribing Latin manuscripts at the Abbey Library of s t. Gall in s witzerland and was hoping to leverage technology to improve this process.

“There was a lot of generosity and sharing [of] information on both sides,” s cheirer said. “We didn’t know basically anything about manuscript studies when we started this project, but on the other side, the humanists didn’t know anything about AI … s o we needed each other.” b ut s cheirer has even more ambitious ideas for how AI vision can help the humanities. s cheirer mentioned the concept of “distant reading,” where researchers could have AI analyze thousands of works to discern broad patterns or connections between these texts and the knowledge within them.

After seeing positive results with Latin, s cheirer has applied similar computer vision programs to transcribing or translating materials in other languages. Projects have revolved around more mainstream languages like French or forms of e nglish, but also obscure dialects like Ge’ez, an ancient e thiopian script.

“You can’t read all the books, but what if you could summarize information in many thousands of books?” s cheirer said. “ c an you see trends over time? What ideas come into fashion? What ideas are around for a little while and go out of fashion? [It’s] like the historical progression of knowledge, right? You can use textual analysis algorithms to do that.”

As interest increases in the emerging field of digital humanities, s cheirer sees AI as a tool that will expand the possibilities of humanities scholarship.

“It opens up new modes of inquiry,” s cheirer said. “We’re able to pose questions that we couldn’t pose before and start to tackle them using these computational tools.”

Putting the “science” in computer science

Jian-xun Wang, Assistant Professor, Aerospace and m echanical e ngineering

That AI would have a multitude of applications in the sciences is perhaps not unexpected. c omputer science is, after all, a science itself with a close relationship to other formal and natural fields.

Indeed, Jian-xun Wang, assistant professor of aerospace and mechanical engineering, finds that in his research revolving around AI in scientific applications, it can be difficult to discern whether one is a computer scientist, another kind of scientist or all of the above. b ut we also closely collaborate with pure domain/ computer scientists to have deep expertise in each field. We view ourselves as the bridge.”

“In scientific [machine learning], it is hard to define computer scientists and domain scientists,” Wang wrote in an email. “ m ost of the time, we are both, since you need to understand the physics behind the problem and also machine learning techniques to create new scientific AI models.

Wang’s “ c omputational m echanics & s cientific Artificial Intelligence” ( c o ms AIL) Lab at n otre d ame works on a range of projects at the intersection of AI and scientific research.

A significant portion of Wang’s research has, dating back to his Ph d days, involved using AI to model complex physical phenomena like fluids. In these cases, AI models are advancing research related to physics, but physics is also advancing the development of AI by providing knowledge and principles upon which AI models can be improved. In this sense, there is a reciprocity to the research — computer science and natural science are mutually advanced.

Physics-informed AI programs have numerous applications in their own right. The c o ms AIL Lab has used AI to model fluids in engineering applications, turbulent water flows related to ships in maritime or naval contexts, and blood flows in cardiovascular systems.

While direct scientific applications are helpful, Wang indicated that AI’s most disruptive impact on science may come in a different, perhaps unexpected way. b y using AI not for “doing” science per se but for reading papers and conducting literature reviews, scientists could — in a manner analogous to the “distant reading” mentioned above — drastically enhance their ability to extract knowledge, make connections and advance research in a transformative fashion.

“AI has the potential to revolutionize the way scientific research is conducted, from streamlining data collection and analysis to generating new scientific hypotheses,” Wang wrote.

“AI algorithms can help researchers process and analyze large data sets, identify patterns and relationships that would be difficult for humans to detect and make predictions that can guide future experiments.” h owever, Wang noted that AI disruption is not inevitable. r esearchers will need to make an active effort to ensure AI systems are functionally safe and responsibly applied.

While the use of AI for textual research can help any scientific field, Wang mentioned genomics, drug development and climate forecasting as areas that may especially benefit from AI readers combing over massive amounts of data and recognizing patterns.

“There will need to be continued investment in the development and application of AI technologies in scientific research to ensure that their full potential is realized,” Wang wrote.

Using ai to track online violence

Tim Weninger, Frank m . Freimann Associate Professor of e ngineering, c omputer s cience and e ngineering

When Tim Weninger, Frank m . Freimann associate professor of engineering, first visited U s AI d (U. s Agency for International d evelopment), he said he felt a bit out of his element.

“I was the only computer scientist within ten miles of that place,” Weninger said. “ c omputer scientists are more interested in making the latest startup than going and engaging with the s tate d epartment or with U s AI d s o I was a fish out of water.” r ather than going the stereotypical startup route, Weninger wanted to apply his knowledge of computer science and AI to solve problems related to peace and violence. n ow, Weninger and colleagues from across campus work on a U s AI d -funded project that tracks indicators of violence on social media.

The project originated in the Indo-Pacific region where Weninger explained that “Facebook is the internet” — the primary source for communicating with others, for consuming media and for accessing information. In this region, as in many parts of the world, Facebook has proven to be a platform for propagating misinformation via altered images and videos, sometimes with violent connotations.

That’s where AI comes in. Weninger and his team created a program to comb social media for doctored content that promotes violence or hatred. Their AI can sift through millions of images per day, a task impossible for humans, and identify warning signs of violence.

“We can learn a lot about a country and a culture and a people and what they’re afraid of by looking and watching how they manipulate images,” Weninger said. “With AI tools, I can discover that and tell us the story that is kind of emerging.” d espite being a computer scientist, Weninger’s roster of postdocs includes people from diverse backgrounds, like peace studies. While Weninger believes in AI’s power to solve social scientific or political problems, he also recognizes that computer scientists need to collaborate to achieve impact in these spaces.

After success in Indonesia with their AI, Weninger and his team are expanding into other countries, most recently Ukraine and France. This initial project with U s AI d has also led to more AI-driven collaborations planned for the future.

“ c omputer scientists pretend that we can solve the world’s problems through technology, but we can’t.” Weninger said. “We can bring technology to bear on problems but we need subject matter experts who can guide us in these problems. That’s why I have hired folks from [the Kroc Institute] or from [the Pulte Institute] — because they’re experts.” m oving forward, Weninger thinks AI will have a disruptive role in the social sciences. And he sees n otre d ame as a leader in advancing new AI applications that focus on helping people in the way that his work has done.

“We have a different tact here at n otre d ame,” Weninger said. “It’s about the ‘why’ of [AI], the ‘what are we going to use [AI] to solve.’ And the problems or needs we are going to solve are not necessarily the next Uber, but the next way of helping people better the environment or better how they communicate or better the democratic principles of their local community. That’s the distinctively n otre d ame part of this.”

Contact Spencer Kelly at skelly25@nd.edu

insidE ColUmn