Nudging, Structured Techniques

TRACE Project Draws Funding From Intelligence Agency Contract Multi-University, Multi-Disciplinary Effort Developing Web App to Enhance Critical Analysis and Creative Thinking

Jennifer StromerGalley

Carsten Oesterlund

Nancy McCracken

Aproject supported by a contract from the CREATE (Crowdsourcing Evidence, Reasoning, Argumentation, Thinking and Evaluation) program of the Intelligence Advanced Research Projects Activity, an arm of the Office for the Director of National Intelligence, is underway at the iSchool’s Center for Computational Data Sciences research hub.

Multi-disciplinary in orientation and multidimensional in focus, the project involves a number of principal investigators with diverse specialties based in several universities around the U.S. Also participating is the Syracuse-based not-for-profit research and development company, SRC, Inc. Called TRACE (Trackable Reasoning and Analysis for Collaboration and Evaluation), the project aims to improve reasoning and intelligence analysis through the development of a web-based application that leverages the use of structured techniques, crowdsourcing and smart nudging to enhance analysts’ problem-solving abilities and foster creative thinking. Work builds on a careful analysis of the weaknesses of current approaches and varied ways to enhance critical decision-making.

It is led by iSchool Professor Jennifer StromerGalley, who is joined by a team of principal investigators and a number of researchers, practitioners and graduate students with unique abilities and expertise. Their disciplines include human-computer interaction, deliberation, crowdsourcing, game and experimental design, interface and software design, cognitive and decision sciences and computational techniques.

For its first phase, the project is worth $5,215,441 in funding, $1 million of which was awarded in 2017. Supporting ST-guided Analysis TRACE

Mystery-solving Structures Automated Templates

Crowdsourcing User Network

A n a l y t i c R e p o r t P r o d u c t Smart Nudging Structured Templates

GOALS: ENHANCE REASONING AND COMMUNICATION Researchers are investigating the role of several techniques for enhancing reasoning while also promoting better communication and discussion among work groups. Their aim is to improve the division of labor and reduce typical communication and interaction errors in order to help intelligence analysts accurately and efficiently reason through complex tasks to produce clear, well-supported intelligence products.

Game-based principles of human-computer interaction are being applied to create an engaging and intuitive solution that promotes efficiency, accuracy and clarity in analysis. The TRACE system also uses background software processes (such as machine learning, simple decision trees, and advanced natural language processing) and it adopts responsive and just-in-time mechanisms.

“Our goal is to create a reasoning and reporting application that is not only effective but also appealing to users by making the process intriguing and fun while not interfering with their natural

Members of the TRACE project team meet in Syracuse in June, 2017.

“Our goal is to create a reasoning and reporting application that is not only effective but also appealing to users by making the process intriguing and fun while not interfering with their natural reasoning and writing abilities.”

— JENNIFER STROMER-GALLEY, PROFESSOR, CCDS DIRECTOR

reasoning and writing abilities,” said StromerGalley. “What makes this project unique is that we are rigorously testing every aspect of our application using experimental research methods. When this project is done we will have a proven effective tool for people to use.”

EXAMINING DIGITAL TOOLS AND TACTICS The TRACE team is conducting a series of experiments in each of the project’s three phases to discern different techniques to identify the best approach to improve human reasoning through the use of digital tools. Those include testing and comparing the effects of different analytic techniques and how varied interactions affect reasoning and reporting to improve user-interface and application design. They note that possible benefits of a platform such as TRACE may go beyond the intelligence com - munity to provide guidance to those interested in improving group communication and teamwork. The team is also exploring commercial technology applications.

In addition to Stromer-Galley, principal investigators include James Folkestad of Colorado State University; Kate Kenski, of the University of Arizona; iSchool Associate Professor Carsten Oesterlund and as co-PI, Nancy McCracken, iSchool research associate professor. Additional PIs are: Lael Schooler and David Kellen, faculty from Syracuse University’s Arts and Sciences Department of Psychology; Ben Clegg, of the Colorado State University Department of Psychology; and Brian McKernan of Sage College of Albany.

The team is supported by iSchool Research Associate Professor Nancy McCracken,postdoctoral researcher Patrícia Rossini andCCDS research staff member Yatish Hegde.Two experts in intelligence analysis from SRC Inc., Roc Myers and Sarah Taylor, plus Deborah Plochocki, a PMP-certified technical manager at SRC, also join the group. n