Center for Identification Technology Research (CITeR): 2002-2022 Anniversary Issue

2002-22 | Anniversary Issue

20 YEARS OF THE CITeR COMMUNITY ENABLING CUTTING-EDGE RESEARCH Science of Biometrics | Novel Biometrics | Multispectral Biometrics | Behavioral Biometrics | Trust & Privacy | Video Analytics | Cybersecurity | Mobile & Computing | Fusion & Performance

A NATIONAL SCIENCE FOUNDATION INDUSTRY/UNIVERSITY COOPERATIVE RESEARCH CENTER

THE 20TH YEAR

CITeR Affiliates: Then and Now

Since awarding the first research grants in spring 2002, CITeR — a National Science Foundation (NSF) Industry/ University Cooperative Research Center (IUCRC) comprising faculty, researchers and students — has been serving the needs of industrial and government affiliates in identity science and biometrics.

Founding Affiliates, 2002: Department of Defense — Office/Biometrics Fusion Center Federal Bureau of Investigation (FBI) National Security Agency (NSA) The Biometrics Foundation Viisage

The Last 10 Years ...

68

PhD graduates involved in CITeR research

55

master’s graduates involved in CITeR research

91

bachelor’s graduates involved in CITeR research

72

faculty engaged in CITeR research

190

research projects awarded by CITeR

149

CITeR webinars given by CITeR researchers and students

403

publications by CITeR researchers and students

10K

citations of CITeR researcher and student publications

+

c i t e r. c l a r k s o n . e d u

Current Members, 2022: ACV Auctions Army Futures Command — Combat Capabilities Development Command Armaments Center (CCDC — Armaments) Cyber Street Solutions Athena Sciences Aware Inc. Defence Research and Development Canada (DRDC) Defense Forensic Science Center (DFSC) Department of Defense — Defense Forensics and Biometrics Agency (DFBA) Department of Homeland Security — Office of Biometric Identity Management (OBIM) Department of Homeland Security — Science and Technology Directorate (S&T) Federal Bureau of Investigation (FBI) IDEMIA National Security Agency Precise Biometrics Public Safety Canada Qualcomm Incorporated SICPA Synolo Biometrics Inc. TECH5 Veridium Xator Corporation

On the cover: Thank you to the CITeR researchers, pictured in the collage, who have supported this work over the last 20 years!

DIRECTOR’S MESSAGE

Message From the Director Congratulations to all on CITeR’s 20th year in 2022! CITeR was founded in 2001 and held the first meeting in 2002. At that time, Then Lawrence Hornak at West Virginia University Now Then (WVU) was the founding director and was Lawrence Hornak, Founding Director (2001-10) joined by Anil Jain from Michigan State University and Jim Wayman from San Jose State University. WVU faculty included Bojan Cukic, Michael Schuckers, Tim Norman and myself. In 2002, Bojan Cukic moved to the co-director role. Finally, in 2011, I became CITeR director as a professor at Clarkson University (2011- ). Over time, CITeR also added University of Arizona (2006-17), University at Buffalo (2013- ), Michigan State University (2019- ) and an international site, Idiap (2019- ) in Martigny, Switzerland. I am tremendously lucky to be part of an established center with a committed affiliate board and to be able to “stand on the shoulders of giants.”

Over the years, I have been able to see the center from multiple perspectives and understand the benefits of involvement, including building Then Now relationships with government and industry and addressing research Bojan Cukic, Co-Director and Director (2002-14) challenges. The practical aspect of the center appealed to me — solving real-world problems that impact border security, defense, benefits distribution, consumer electronics, e-commerce and many others. At points, we wondered whether this research would become a “solved” problem, and then new challenges would emerge: spoof detection, biometric cryptography, altered finger detection, biometrics at a distance, biometric permanence, etc. And now here we are in 2022, with additional challenges such as face morphing, deepfakes, bias, template security, biometrics in children and many others.

Please enjoy this issue, which highlights our research and impact. It is my pleasure to be part of this community, and I look forward to working with everyone in the next decade!

Now

Stephanie Schuckers, Director of the Center for Identification Technology Research (CITeR)

Information sharing evolution from binder to CD to datastick to cloud

C I Te R | 1

FACES OF CITER

Distinguished CITeR Researcher Anil Jain, Michigan State University “On behalf of the CITeR directors, I am honored to highlight Dr. Anil Jain. Jain played an important role in the eventual establishment of CITeR through his initial interactions with WVU faculty members Larry Hornak, George Trapp and Bojan Cukic. Jain served in the AAAS advisory committee of a 1998 West Virginia EPSCoR Award that focused on identification technologies, providing valuable advice and guidance to the project team, which led to the CITeR planning meeting in 2001. As a PhD student, I remember traveling to Morgantown in 2002 with Jain and presenting our research on multimodal biometrics at the first CITeR meeting! To this day — 20 years later — Jain continues to contribute to the mission of CITeR, both by performing exceptionally strong research and mentoring numerous faculty and students (and directors!). Thanks to Anil Jain’s mentorship and support, my personal story includes transitioning from a CITeR student to CITeR faculty to a CITeR director.”

— Arun Ross, Professor | Cillag Endowed Chair, MSU Anil Jain is a University Distinguished Professor at Michigan State University and a highly cited researcher with an h-index of 202 at the time of this writing. In 2016, he was elected to the U.S. National Academy of Engineering and Indian National Academy of Engineering “for contributions to the engineering and practice of biometrics.” He has received numerous high profile awards and is a Fellow of ACM, IEEE, AAAS, IAPR and SPIE. He served as a member of the U.S. National Academies panels on Information Technology, Whither Biometrics and Improvised Explosive Devices. c i t e r. c l a r k s o n . e d u

Affiliate Advisory Board Chairs: Past and Present Chris Chamberlin DHS, Office of Biometric Identity Management (OBIM) Futures Identity Implementor / Current Chair, NSF CITeR “CITeR is all about discovering the future of biometrics and identity. At DHS/OBIM, we have found that the more we look into the CITeR well, the more we are amazed by the depth that CITeR researchers discover and share.

These findings strengthen the entire affiliate community. Thanks to CITeR for 20 great years, and 20 more to come!”

Chris Chamberlin

Kody West Senior Staff System Architect at Peraton Former Chair, NSF CITeR “20 years! Congrats on this significant milestone of 20 years of contributions to the advancement of biometrics and identification. I had the privilege of working with CITeR over the course of approximately seven years. During that time, I found the most enduring and important aspect of the group was the community it created and the sharing of ideas Kody West and concepts across academia, industry and the users. The blending of real-world user needs, advanced concepts, research ideas and constraints of application is unique and quite valuable to everyone involved. I’m thrilled to be able to be a small part of this exciting celebration and wish everyone involved continued success! Cheers!”

FACES OF CITER

Robert DelZoppo AVP, Strategic Technology Programs, SRC Inc. (retired) Former Chair, NSF CITeR “Working with CITeR has been one of the highlights of my career. The NSF IUCRC model provides a framework for conducting precompetitive fundamental and applied research of shared interest to industry and government organizations, and CITeR is an exemplar for this model.

The broad scope and challenges of the identification problem space; engagement of the academic, government and industry partners; research portfolio; and dedicated leadership team together provide a highly collaborative environment for impactful research.”

Robert DelZoppo

Chris Miles Deputy Director for Standards Integration and Application DHS, Science and Technology (retired) Former Chair, NSF CITeR “Even from poolside on a cruise ship off the island of St. Croix, I still think CITeR is one of the best partnerships available to industry, government or universities. I’m so proud of what we were able to accomplish and research, and provide leadership towards while I was involved in CITeR. Retirement’s great, but I’m still very proud of what we did together during the years I was involved in CITeR.”

Chris Miles

Rick Lazarick Chief Scientist, Biometrics, Computer Sciences Corporation (retired) Former Chair, NSF CITeR “CITeR provided the unique and valued opportunity to interact directly and personally with the leading researchers in the field of biometrics, resulting in very strong professional and personal relationships. As intended, the leading edge topics researched by CITeR were first available to the sponsoring organizations.

Several of the young student researchers became valued employees and coworkers with the sponsors, one of the founding concepts and benefits of CITeR.”

Rick Lazarick

Jeff Dunn Technical Director for Biometrics National Security Agency (retired) Former Chair, NSF CITeR “CITeR was founded to encourage research in biometric and other identification technologies. By any measure, CITeR has far exceeded expectations and, for 20 years, has fostered collaboration among government, industry and academia with many, many successful results.”

Jeff Dunn

C I Te R | 3

FACES OF CITER - ALUMNI HIGHLIGHTS

University at Buffalo Nishant Sankaran, PhD, Computer Science and Engineering, 2021 Applied Scientist, Amazon Web Services “My first project in CITeR was ‘Probabilistic Identity Reasoning in Camera Networks,’ which introduced me to face recognition.

Subsequent CITeR projects relating to face recognition and fusion, and interactions with the CITeR community during review meetings, greatly helped me contribute to the IARPA Janus program for unconstrained face recognition. The foundation of my thesis on fusion of deep features was the research I conducted in face recognition as part of CITeR, and I am indebted to CITeR for it.”

West Virginia University Simona Crihalmeanu, PhD, Electrical Engineering, Biometrics, 2012 Lead Engineer/Biometrics, Research Scientist, Booz Allen Hamilton “My involvement with CITeR-funded projects started in 2006 while working as a research associate in the CITeR laboratory and enrolled in the PhD program at WVU. As a student, I focused on multispectral ocular biometrics. I was involved in many aspects of the research that taught me how to initiate and bring projects to completion. I was responsible for the design, implementation and maintenance of the CITeR biometric database; the release of the biometric datasets to the national and international community; and IRB documentation for CITeR funded projects, etc. After 14 years in the academic environment, I continued my career in industry as a researcher at EyeVerify, working on algorithms to detect presentation attacks in mobile biometrics.”

Nathan Kalka, PhD, Computer Science and Engineering, 2012 Senior Computer Vision and Machine Learning Scientist, Noblis “Throughout my graduate years at WVU, CITeR provided me with many opportunities to solve challenging, relevant and high-impact problems in the field of biometrics. CITeR-funded research projects enabled me to hone and develop my skills in computer vision, machine learning and statistical pattern recognition. Beyond the technical research, I also gained valuable experience in all life cycles of project management, from proposal writing to managing project execution. Arguably most importantly, CITeR provided me with many opportunities for networking and fostering relationships with academic, industry and government partners. The knowledge, experience and relationships have been invaluable and ultimately helped shape and define my professional growth in the competitive biometrics industry.”

Clarkson University Bozhao Tan, PhD, Electrical Engineering, 2008 Senior Manager, Data Science (Speech, NLP), Capital One “I was very lucky to start my research career at Clarkson/CITeR in 2004 with Dr. Schuckers; through projects on fingerprint liveness and multimodal fusion to reduce spoofing vulnerability, I gained experience on how to conduct research and transition from research to real products. With 13 years in industry, I’m happy to see the research we explored over 10 years ago has become must-haves in current biometrics systems, like face and fingerprint liveness. Today, deepfake generation has grown fast due to the advancement of deep-learning techniques; to counterattack, we must research ways to leveraging multiple data sources related to face, voice and video movement. Additionally, in industry, I find that end users care about privacy while using biometrics and wake word detection in electronic devices. People have high expectations regarding both convenience and security/privacy concerns.” c i t e r. c l a r k s o n . e d u

NATIONAL EVENTS / FACES OF CITER - ALUMNI HIGHLIGHTS

Michigan State University Sunpreet Arora, PhD, Computer Science and Engineering, 2016 Senior Research Scientist, VISA “We are in the ‘industrial era’ of authentication powered by biometrics. Similar to how the economy transitioned from being largely manual and agrarian-based to being automated by machine manufacturing, the authentication status quo in every industry, be it healthcare, transportation or payment, is being transformed to a more seamless and secure approach using biometrics. It is an exciting time for the biometrics research community as a multitude of practical challenges, ranging from privacy, security and unconstrained recognition to bias and fairness, need to be addressed for trustworthy use of biometric technology.”

Challenge Problem Workshops These events offer the opportunity to share, learn and discuss current issues in the biometric arena. Affiliate members often host, allowing the entire CITeR community to learn more about our members operations and challenges. The workshops were paused for COVID-19, but we look forward to resuming the efforts!

2019, 2015 — FBI Center of Excellence 2019 — DHS/Maryland Test Facility 2018 — Defense Forensic Science Center (DFSC) 2016 — Lower Manhattan Security Coordination Center in New York City 2016 — DHS/Peace Bridge in Buffalo, New York

Congressional/UN Testimonies

Screen capture: House Science Committee — Democrats

Dr. Stephanie A.C. Schuckers, May 2013 | U.S. House of Representatives | Committee on Science, Space and Technology | Subcommittee on Research and Subcommittee on Technology: The Current and Future Applications of Biometric Technologies Dr. David Doermann, June 2019 | U.S. House of Representatives | Permanent Select Committee on Intelligence: National Security Challenges of Artificial Intelligence, Manipulated Media and “Deepfakes” Dr. Arun Ross, May 2013 | United Nations Counter-Terrorism Committee (CTC) | UN Headquarters: Countering Terrorism Through the Use of New Communications and Information Technologies Dr. Siwei Lyu, September 2019 | U.S. House of Representatives | House Committee on Science, Space and Technology: Online Impostors and Disinformation C I Te R | 5

RESEARCH HIGHLIGHTS

Biometric Aging in Children and Adults Anil Jain, Stephanie Schuckers, Laura Holsopple

Age 6

Age 7

Age 8

There has been limited study of uniqueness and permanence, despite their being the critical core principles for biometric recognition. CITeR has funded fundamental scientific studies of permanence, considering both aging in adults and children. Foundational studies of fingerprints and faces were published in 2014 and 2015, respectively, and found that despite a decrease in match scores over time, false non-matches stayed high over a period of 12-15 years for both faces and fingerprints in adults.

Furthermore, CITeR researchers have studied biometric recognition in children, which has applications in areas including immigration, refugee efforts and distribution of benefits. This includes a biometrics collection effort in children for nine sessions over six years, which created one of the only longitudinal research datasets of its kind from children for fingers, faces, irises, feet, ears and voices. This dataset has led to advances in our understanding of iris recognition in children. The significant finding is that irises are stable for three years in children aged 4 to 11, and face recognition algorithms fail over three years, except for the most recent state-of-the-art deep-learning algorithms. In other research funded by the Gates Foundation, fingerprint biometrics have been explored for infant recognition.

Face Morph Generation and Detection Nasser Nasrabadi, Jeremy Dawson, Xin Li, Chen Liu, David Doermann, Srirangaraj Setlur, Arun Ross

Age 9

c i t e r. c l a r k s o n . e d u

Face morphing is the process of combining two or more subjects in an image to create a new identity that contains features of both individuals. Morphed images can fool facial recognition systems into falsely accepting multiple people, leading to failures in national security and border control applications. In the last three years, CITeR researchers have developed several single and differential morph detectors based on deep-learning architectures to circumvent the problem of morph attacks. Single morph detectors are used during passport applications to authenticate if the submitted passport photo is a real or morphed image, and differential detectors are used to validate if the photo of the passport holder is the

CITER PROJECTS

same as the photo in the passport. Recently, a deep-learning-based morph detector developed by CITeR researchers, referred to as WVUsingle-001 in the NIST report, has been evaluated by the NIST morph detection challenge vendor test and has achieved first- or second-rank performance on different morph detection scenarios (i.e., low-quality, print & scan and high-quality and low-resolution morph datasets). The NIST evaluation reported that the WVUsingle-001 detector has the first rank in detecting Automated Global Morphs, MIPGAN-II and print & scan morph image datasets, and has achieved the second rank on Automated Local Morph, Visa-Border and Twente morph image datasets. Furthermore, as morphed image synthesis becomes easier, expanding the datasets available to the research community is vital to help combat this dilemma. Using face photos of look-alike individuals, CITeR researchers have generated morph databases based on facial landmarks (i.e., OpenCV, Facemorpher), wavelet-based features and deep-learningbased methods (i.e., StyleGAN-2 and MIPGAN-II). Additionally, to smooth out StyleGAN-2-based generated morphs the morphing artifacts to fool the detectors, print & scan, compressed and adversarially-perturbed morph databases were created. For all generated morph datasets, a vulnerability study was performed to quantify the percentage of morphing attacks that can fool a COTS and our own face matcher. The vulnerability study based on Mated Morph Presentation Match Rate is above 95% for all our morph datasets.

Cross-Spectral Face Recognition Nasser Nasrabadi, Jeremy Dawson, Thirimachos Bourlai, Natalia Schmid, Arun Ross The large gap between images captured in different spectra makes heterogeneous face recognition (HFR) or cross-spectral iris recognition challenging.

In typical HFR scenarios, a probe in the infrared domain (i.e., NIR and thermal) or a hand-drawn facial sketch is to be matched against a large gallery of visible faces.

In CITeR, researchers studied hand-crafted feature descriptors as well as deeplearning-based features to reduce the cross-modality between visible and NIR, SWIR, MWIR or LWIR images. Deep coupled neural network architectures based on the Generative Adversarial Network (GAN) model were implemented. A high matching accuracy between visible and NIR/SWIR faces was reported on the WVU cross-spectral

C I Te R | 7

RESEARCH HIGHLIGHTS

database. However, in the case of matching LWIR against visible face imagery, the matching performance was lower, with a rank-one identification accuracy of 88.57%. Using a polarimetric thermal database, CITeR researchers also demonstrated that the matching accuracy between visible and polarimetric thermal probes can be improved to as high as 94.08% for rank-one identification by exploiting the thermal and polarization state information. Furthermore, CITeR researchers investigated jointly using super-resolution and cross-spectral translation for matching high-resolution visible iris probes against low-resolution NIR legacy databases. A matching accuracy of 84.98% at a false alarm rate FAR=0.001 was achieved.

NIR

Visible

SWIR

MWIR

Low atmospheric transmittance window

SWIR

LWIR

MWIR

Face images across different imaging spectrums

c i t e r. c l a r k s o n . e d u

VLWIR

LWIR

Noncontact Fingerprint Recognition Jeremy Dawson, Nasser Nasrabadi, Stephanie Schuckers, Nalini Ratha, Anil Jain

Traditional livescan fingerprint images are acquired by pressing or rolling a finger against a surface, or platen. Although mature and well-understood, livescan images often require a stand-alone sensor unit, and the emergence of COVID-19 has increased hygiene concerns in fingerprinting. Contactless fingerprinting devices and smartphone apps have been introduced to overcome the drawbacks of livescan technologies. However, interoperability issues have arisen when comparing contactless fingerphotos and contact-based galleries that can reduce matching accuracy, such as elastic and photometric distortions, along with operational challenges in collecting fingerphotos, such as motion blur, focus blur, nonuniform lighting, etc. These challenges are compounded by the fact that existing fingerprint quality metrics, designed for contact images, are not directly applicable to contactless fingerprints. To overcome these issues, CITeR researchers have developed deep-learning methods of removing elastic and photometric distortion from livescan and contactless images to match fingerphotos to legacy galleries of contact images in a common feature subspace, improving match accuracy above what is achievable using only simple “contact-equivalent” images. Researchers have also explored the development of new contactless fingerphoto apps that can be used by untrained operators, and also analyzed presentation attack detection (spoofing) for contactless fingerprints.

CITER PROJECTS

Recent CITeR projects have focused on using deep-learning approaches to de-blur fingerphotos and developing a fingerprint quality metric that can be applied to fingerphotos and their resulting contact-equivalent images.

Template Security and Privacy Arun Ross, Anil Jain, Vishnu Boddeti, Matthew Valenti, Nasser Nasrabadi

 

 

A number of research projects in CITeR have focused on extending security and privacy to biometric systems. A template denotes the set of features (such as fingerprint minutiae) that is stored in a database.

An adversary can potentially steal this template or modify its contents, thereby undermining the integrity of the biometric system. Early CITeR research on template protection included the use of Fuzzy Vault and Fuzzy Commitment schemes to design fingerprint cryptosystems in which biometric matching was implemented in the encrypted domain. Similar methods were developed for multimodal biometric templates. More recently, a hybrid secure architecture based on cancelable biometrics and secure sketches was used in tandem with “deep hashing” to generate a multimodal secure sketch that was cryptographically hashed to generate a secure multimodal template for the Enrollment Database Unencrypted Face Recognition System ID:01 face and iris. Other work includes the ID:02 ID:03 development of HERS (Homomorphically Template ...   Impersonation?

Encrypted Representation Search) — a Reconstruction Gender?

Feature Algorithm       Age? Extraction   technique that can rapidly match an input Race?  Query Face Reconstructed Stolen image against a database of encrypted  Matcher Face Template templates while retaining identification accuracy. At the same time, a number Hacker of privacy-preserving schemes have been developed. For example, in a recent Encrypted Database Encrypted Face Recognition System project, CITeR researchers developed a ID:01 ID:02 Encrypted ID:03 method to ensure that a face image could Template Template ...   only be used for biometric recognition Privacy Feature   HERS      ?  Preserved  Extraction purposes but not for the purpose of  Query Face extracting sensitive attributes such as Matcher Encrypted Template (Encrypted Domain) Stolen age, gender, race and health cues.

Hacker

C I Te R | 9

RESEARCH HIGHLIGHTS

Bias in Face Recognition Tiago de Freitas Pereira, Sébastien Marcel, Stephanie Schuckers, Jeremy Dawson, Mahesh Banavar

0.12% 0.12%

0.31% 0.19%

Two cases of comparisons (reported in terms of % of similarity) between two different identities for a biased FR system and bias-mitigated FR system

In machine learning, and therefore biometrics, fairness issues arise from the analysis of figures of merit (e.g., accuracy) in specific demographic groups (e.g., gender, ethnicity, race, etc.).

The large-scale deployment of such systems raises the debate about its fairness and its impact on our lives. In the biometrics literature, aspects of fairness have been recently addressed for some biometric traits such as face or voice. Pereira and Marcel proposed a novel figure of merit, the Fairness Discrepancy Rate (FDR), to assess fairness in biometric verification systems.

Assessing fairness with standard figures of merit (DET, ROC) gives the impression that biometric systems are fair while they are not. The FDR assesses fairness by verifying that both FMR and FNMR are equally separable between different demographic groups under the same decision threshold and allows to compare how fair two biometric verification systems are with respect to different demographic attributes for single decision thresholds.

Other CITeR research includes the development of a measure of skin tone based on a single image, which can be used for training fair face recognition systems, particularly when ground truth labels are unavailable. More recently, CITeR research has found gender and racial disparities in commercial and academic voice recognition systems.

Presentation Attack Detection, AKA Liveness Detection Stephanie Schuckers, Lawrence Hornak, Arun Ross, J. Xia, Anil Jain, K.W. Oh Since CITeR’s first project in 2002, CITeR was an early leader in research on presentation attack detection (e.g., liveness), or the detection of attacks using fake biometric samples, commonly called spoofs. As one of the first papers in this area, CITeR’s focus includes “software-based” liveness — that is, based on the biometric image only. The algorithm takes advantage of these differences between live and spoof images through feature extraction and pattern recognition. A distinctive contribution of CITeR research is the c i t e r. c l a r k s o n . e d u

CITER PROJECTS

development of benchmark datasets, competitions, standardization and certification. CITeR researchers and their collaborators hosted a competition series (LivDet), held in 2009, 2011, 2013, 2015, 2017 and 2021, which included sharing a training set of spoof and live images to competitors. The results provided the state-of-the-art performance of algorithms and benchmark datasets for further study. This has also led to contributions to ISO standards and requirements for FIDO Biometric Component Certification. CITeR research has also included novel aspects of hardware using photoacoustic technology and sophisticated spoofs of fingers with pulsing blood using microfluidics and using makeup to defeat face recognition technology. CITeR has also explored novel biometrics that have liveness characteristics built in, like an electrocardiogram and electroencephalogram.

Soft and Novel Biometrics Jun Xia, Wenyao Xu, Srirangaraj Setlur, Nils Napp, Venu Govindaraju Soft biometrics use physical or behavioral traits that can be naturally described to assist in human identification and are often used to supplement traditional biometrics. CITeR researchers have explored a variety of soft biometrics, including recently the use of clothing people wear based on the premise that people typically have a limited distinctive wardrobe. The work demonstrated a novel way of automatically identifying people from still pictures or video footage based on modeling the wardrobe of individuals observed over multiple days of video. CITeR researchers have also explored a novel approach that uses 3D images of finger veins using photoacoustic tomography, an imaging technique that combines light and sound. First, a short-pulsed light illuminates the finger. If the light hits a vein, it gets absorbed and releases a sound wave.

The system then detects that sound with an ultrasound detector and uses it to reconstruct a 3D image of the veins. A new matching algorithm was also developed to match features in 3D space. Tests have shown that the approach is not only feasible but also accurate.

Finger vessel images from eight different subjects, with colors that represent different depths

C I Te R | 11

CITER APPROVED PROPOSALS

CITeR Fall 2020 Approved Proposals – Biometric Data Classification for Large-Scale Database Error Detection and Correction Jeremy Dawson (WVU), Nasser Nasrabadi (WVU) – Deepfake Video Fingerprinting Siwei Lyu (UB), Xin Li (WVU) – Digitally Altered Data: Finding the Original From NearDuplicate Biometric Images Arun Ross (MSU) – High Resolution Face Completion for Authentication Under the COVID-19 Pandemic Gianfranco Doretto (WVU), Don Adjeroh (WVU) – Joint Face Pose Estimation and Frontalization Nasser Nasrabadi (WVU), Jeremy Dawson (WVU) – Leveraging Finger Relationships for 2-finger Authentication on Mobile Devices Sergey Tulyakov (UB) – Multimodal Feature Fusion for Automobile Abnormality Detection Srirangaraj Setlur (UB) – Preserving the Privacy of Face Embeddings in Modern Face Recognition Systems Vedrana Krivokuća Hahn, Sébastien Marcel (Idiap) – Toolkit for Explainable AI in Biometric Recognition Stephanie Schuckers (CU), Mahesh Banavar (CU) – Touchless Fingerprint Biometrics at Border Crossing With User Mobile Phones Nalini Ratha (UB)

CITeR Spring 2021 Approved Proposals – Biometrics at Scale: Generating Large-Scale Synthetic Fingerprint and Iris Datasets Arun Ross (MSU), Anil Jain (MSU) – Biometrics Image Assurance: From Hardware to Software Daqing Hou (CU), Wei Yan (CU), Siwei Lyu (UB), Xiaoming Liu (MSU), David Doermann (UB), Srirangaraj Setlur (UB), Nalini Ratha (UB), Arun Ross (MSU) c i t e r. c l a r k s o n . e d u

– Deep Deblurring of Fingerphotos Captured by Smartphones Jeremy Dawson (WVU) – Detection of Over-Rotated Biometric Images and Incorrect Labeling of Fingerprints Srirangaraj Setlur (UB), Stephanie Schuckers (CU), Faraz Hussain (CU) – Differential Performance Mitigation in Face Recog. Based on Novel Skin Reflectance Estimate Stephanie Schuckers (CU), Mahesh Banavar (CU), Sébastien Marcel (Idiap) – Evaluation of the Performance of Multi-Finger Contactless Fingerprint Matching Jeremy Dawson (WVU) – Explainable Mechanisms for Deep Neural Networks: A Biometrics Perspective Xiaoming Liu (MSU), Nalini Ratha (UB), Vishnu Boddeti (MSU), Arun Ross (MSU) – Fully Homomorphic Encryption in Biometrics Nalini Ratha (UB), Arun Ross (MSU), Vishnu Boddeti (MSU) – Investigate the Effort of Acoustic Coupling on Ultrasonic Fingerprint Imaging Jun Xia (UB) – Multi-Modal Gait and Anthropometric Data Collection Karthik Dantu (UB), Srirangaraj Setlur (UB) – Open-Source Face-Aware Capture with Anti-Spoofing for Securing Passport Photo Capture Masudul Imtiaz (CU), S.M. Safayet Hossain (CU), Keivan Bahmani (CU), Stephanie Schuckers (CU) – Towards the Creation of a Large Dataset of High-Quality Face Morphs Chen Liu (CU), Stephanie Schuckers (CU), Xin Li (WVU), Jeremy Dawson (WVU), Nasser Nasrabadi (WVU), David Doermann (UB), Srirangaraj Setlur (UB), Siwei Lyu (UB), Xiaoming Liu (MSU), Sébastien Marcel (Idiap)

TRANSITIONS/ OUTCOME & HIGHLIGHTS FINANCIALS

Intellectual Property and Transitions Intellectual property resulting from CITeR projects is governed by the CITeR Affiliate Agreement, which provides for a nonexclusive, royalty-free license to patented technology, provided the interested affiliates pay for costs associated with drafting, filing and prosecuting the patent. CITeR research is often foundational. Not all projects lead to intellectual property. Anecdotally, affiliates have used CITeR’s foundational work to develop their own internal intellectual property, building on CITeR public domain research. Early awareness of research advances, in some cases up to 18 months or more prior to publication, allows affiliates to position their own intellectual property in advance of the marketplace. CITeR intellectual property examples are as follows: Fingerprint Liveness Detection Software CITeR researchers from Clarkson and WVU created fingerprint liveness algorithms designed to minimize the risk of spoofing.

Algorithms were based on the distinctive spatial texture characteristics related to live and spoof fingerprints. The algorithms were licensed to NexID Biometrics LLC, a Clarkson and WVU spinout company.

Precise Biometrics purchased NexID Biometrics in 2017.

Altered Fingerprint Detection MSU researchers developed an innovative approach for automatically detecting altered fingerprints based on pattern analysis techniques and mathematical modeling of fingerprints. The technology was licensed by MSU to Morpho (Safran Group) in 2011.

Conjunctival Vascular Recognition CITeR researchers from the University of Missouri-Kansas City and WVU created software for biometric recognition using the vasculature seen on the white of the eye. The technology was licensed to EyeVerify LLC, a venture capital-funded startup founded 2012 and purchased by Ant Group in 2016, also doing business as Zoloz.

CITeR Income Summary 2017-21 $2,000,000 $1,800,000 $1,600,000 $1,400,000 $1,200,000 $1,000,000 $800,000 $600,000

$50K

$400,000

affiliate membership gains access to the community and a portfolio of in projects per year

$1.7M

$200,000 $0

2017

2018

2019 2020 2021 NSF IUCRC Affiliate $

C I Te R | 13

citer.clarkson.edu Contacts

Affiliate Advisory Board Chair

Clarkson University Dr. Stephanie Schuckers Director 315-268-6536 sschucke@clarkson.edu

Laura Holsopple Managing Director 315-268-2134 lholsopp@clarkson.edu

West Virginia University Dr. Matthew Valenti Site Director 304-293-9139 matthew.valenti@mail.wvu.edu

Dr. Nasser Nasrabadi Site Co-Director 304-293-4815 nasser.nasrabadi@mail.wvu.edu

University at Buffalo Dr. Venu Govindaraju Site Director 716-645-3321 venu@cubs.buffalo.edu

Srirangaraj Setlur Site Co-Director 716-645-1568 setlur@buffalo.edu

Michigan State University Dr. Arun Ross Site Director 517-353-9731 rossarun@cse.msu.edu Idiap Research Institute (international site) Dr. Sébastien Marcel Site Director +41 27 721 77 27 marcel@idiap.ch

Chris Chamberlin Project Manager Office of Biometric Identity Management Department of Homeland Security Affiliate Executive Committee Terry Riopka Senior Director of Research AWARE Inc. Collin D’Souza Senior Research Engineer in Biometrics/Computer Vision Qualcomm Arun Vemury Director, Biometric and Identity Technology Center U.S. Department of Homeland Security Brian Green S&T Program Manager, Border Security and Biometrics Defence Research and Development Canada Department of National Defence, Government of Canada