CNSE: Pioneering Artificial Intelligence Research

The College of Nanotechnology, Science, and Engineering

Md.

Md.

Sukanta

Ming

Kathleen

Michael

Project Summary:

CubeSat-based radiometer constellations bring unique challenges in terms of their calibration as they usually use vicarious targets for calibration and are susceptible to ambient conditions Also, an autonomous real-time (inter)calibration of such constellations rather than post-processing (re)calibration to fully utilize their potential (promptly reacting to critical observations) is desirable To address these challenges, the overarching goal of this research project is to develop a framework named ACCURACy to calibrate constellations of identical CubeSat microwave radiometers in their entirety In pursuit of this goal, the specific objectives are to:

• Establish a machine learning based algorithm to gather constellation radiometers in similar states into distinct and timeadaptive clusters,

• Facilitate calibration data sharing within each cluster,

• Construct an intelligent and adjustable cluster-level absolute calibration structure to quantify and minimize calibration errors and uncertainties,

• Study data processing and transmission requirements, as well as the scalability of the ACCURACy framework for various constellation sizes.

Major Accomplishments:

• Analysis of several CubeSat radiometer datasets have revealed that radiometer calibration characteristics depend on instrument temperatures, temperature gradients, and age

• Analytical relationships between calibration errors & uncertainties and radiometer characteristics have been derived as a function of calibration target temperatures and observation times for autoregressive gain processes

• Coding of the ACCURACy framework has been completed.

• A virtual CubeSat constellation simulator has been developed to test ACCURACy and a digital radiometer hardware prototype is being built to validate the feasibility of the framework

Key Publications:

[1] M Aksoy and J W Bradburn, "A Novel Calibration Framework for Cubesat Radiometer Constellations," IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 2022, pp. 4292-4295.

[2] M Aksoy, H Rajabi, P E Racette and J Bradburn, "Analysis of Nonstationary Radiometer Gain Using Ensemble Detection," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 2807-2818, 2020.

[3] K. J. Coakley, J. Splett, D. Walker, M. Aksoy and P. Racette, "Microwave Radiometer Instability Due to Infrequent Calibration," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 3281-3290, 2020.

This work was supported by an Early Career Faculty grant (Grant #: 80NSSC20K0067) from NASA’s Space Technology Research Grants Program

Website: https://www.albany.edu/ece/microwave-remote-sensing-laboratory

Fig 1 ACCURACy Framework to Calibrate Constellations of CubeSat Radiometers by (1) Clustering radiometers in similar states, (2) creating shared vicarious calibration data pools for each cluster, and (3) implementing cluster-level, adjustable, multi-point absolute calibration

Fig. 2. Accuracy and uncertainty of the calibrated antenna temperatures of a radiometer in a constellation simulation using ACCURACy versus (1) state-of-the-art intercalibration based on overlapping measurements of the constellation members and (2) individual baseline calibration.

Mustafa Aksoy

Assistant Professor, Department of Electrical and Computer Engineering maksoy@albany.edu

Hey, Can You All Look Here? AI-enabled Gaze Uniformity in Group Photos

Project Summary

• Motivation:

• Do you capture photos in group settings?

• Are there multiple individuals taking pictures of the group?

• Do all members of the group face the same camera or gaze in the same direction?

• Objective:

• Developing a method to ensure consistent gazes among subjects in group photos is crucial for determining overall aesthetic appeal.

• Contribution:

• Our proposed method, GARGI (Gaze-Aware Representative Group Image), uses AI algorithms to select group images with optimal gaze uniformity.

• Unlike existing mechanisms, such as those in Apple iPhones, which overlook gaze considerations, especially in group settings, GARGI prioritizes minimizing gaze deviation for each subject relative to their expected gaze directions.

Publications

Highlights

GazaAnalysis App available at: https://apps.apple.com/us/app/gazeanalysis/id6472210088

• O. Kulkarni, S. Arora, A. Mishra, V. K. Singh, and P. K. Atrey. A multi-stage bias reduction framework for eye gaze detection. MIPR'23: IEEE International Conference on Multimedia Information Processing ad Retrieval, Singapore, August 2023.

• O. Kulkarni, S. Arora and P. K. Atrey. GARGI: Selecting gaze-aware representative group image from a live photo. MIPR’22: The 5th IEEE International Conference on Multimedia Information Processing and Retrieval, San Jose, USA, August 2022.

• O. Kulkarni, V. Patil, S. Parikh, S. Arora and P. K. Atrey. Can you all look here? Towards determining gaze uniformity in group images. ISM’20: IEEE International Symposium on Multimedia, Tapei, Taiwan, December 2020.

Acknowledgements

• New York State Grant #940003-01.

Project Summary

The key objectives of this study are to leverage data collected using low-cost PurpleAir sensors for understanding spatiotemporal variation of PM2 5, identify local and distant source impacts. Under the U.S. EPA Region 2’s Participatory Science Equipment Loan Program, we are deploying 20 PurpleAir sensors at 14 libraries/homes, 2 NYSDEC sites and 4 NYS Mesonet weather stations in the State Capital, Hudson Valley, and in New York City. Prior to field deployment, sensors were calibrated by co-locating with a reference monitor at the NYS Department of Environmental Conservation site at Albany for two weeks. Calibrated sensor data will be ingested to an AI/ML model (Artificial Neural Network-ANN) to determine spatiotemporal variability in PM2 5 concentrations and elucidate local vs distant source impacts across neighborhoods.

Data fusion of models, satellites observations, in-situ measurements, and low-cost sensors provide synthesized estimates of PM2 5 at local scale with high resolution

Publications

Highlights

An AI/ML architecture is established to estimate ground-level PM2.5 by fusing numerical models and multi-platform observations.

• Bari, M.A., Eugene, M, Hossain, C.A., L’Hote, M., Patterson, A., Dueker, E., Lu, S., Fitzjarrald, D., 2024. Understanding spatiotemporal variability and local sources of PM2.5 in the New York State Capital, Hudson Valley Region and New York City using PurpleAir sensors. 2024 Air Sensors International Conference, April 20 – May 3, Riverside, California.

Acknowledgements

• Collaborators: Sarah Lu, Atmospheric Science Research Center (ASRC), University at Albany

• Funding sources: EPA PurpleAir Loan Program

www.albany.edu/environmental -sustainable-engineering/faculty/md-aynul-bari

Developing resilience in air quality monitoring tools against AIenhanced adversaries

Project Summary

With the wide-spread adoption of artificial intelligence (AI) in various applications including analyzing air quality, this study focuses on AI’s security impacts on such tools Specifically, this study aims at studying the adversarial effects of AI on air quality monitoring tools as well as building resilient techniques for these tools against AI-enhanced adversaries. To understand the landscape of AI-based threats, the adversarial machine learning threats will be analyzed. This study will identify the characteristics, objectives, and requirements of AI-based threats and classify both adversaries and defenders/users based on these findings. The scope of such AI-based threats, the capabilities of both adversaries and defenders, and other assumptions currently utilized in this domain will be also analyzed The outcome of this research activity will provide a threat model for AI-based adversaries in air quality monitoring along with the development of a security capability for air quality monitoring tools to tackle with this newer generation of adversaries.

Highlights

• Safeguarding air quality monitoring systems

• Tackling with newer generation of AIenhanced adversaries

• Building security solution compatible with sate-of-the-art air quality monitoring tools

Publications

• None

Acknowledgements

• Collaborators: Suryadipta Majumdar, Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Canada

• Funding sources: NSF-IRES (pending)

Predicting dark doldrums using machine learning

Project Summary

As wind and solar power play increasingly important roles in our energy system, unfavorable weather conditions, such as “dark doldrums” (extended calm and cloudy periods, also known as “Dunkelflaute” in Western Europe), will pose ever greater challenges to transmission system operators. Thus, accurate identification and characterization of such events from open data streams (e g , reanalysis, numerical weather prediction, and climate projection) are going to be crucial. Recently, we proposed a two-step, unsupervised deep learning framework [wind and solar network (WISRnet)] to automatically encode spatial patterns of wind speed and insolation and, subsequently, identify dark doldrum events from the encoded patterns. We demonstrated that in lieu of proprietary power production data, the WISRnet framework can identify dark doldrum periods from public-domain meteorological data.

Validation of WISERnet

The shaded area corresponds to dark doldrum periods which are identified by the WISERnet

Publications

Li, Bowen, Sukanta Basu, and Simon J Watson (2022) Automated identification of “Dunkelflaute” events: a convolutional neural network–based autoencoder approach Artificial Intelligence for the Earth Systems 1 4: e220015

Li, Bowen, Sukanta Basu, Simon J Watson, and Herman W J Russchenberg (2021) A brief climatology of Dunkelflaute events over and surrounding the North and Baltic S ea areas Energies 14 20: 6508

Li, Bowen, Sukanta Basu, Simon J. Watson, and Herman W. J. Russchenberg (2021). Mesoscale modeling of a “Dunkelflaute” event. Wind Energy 24.1: 5-23.

Acknowledgements

• Bowen Li (PhD student) and Prof. Simon Watson at Delft University of Technology, the Netherlands.

Sukanta Basu

Project Summary

We have established an ongoing research program on resistive memory devices (aka: RRAM) These metal-insulator-metal (MIM) devices behave similarly to neural synapses, as their “memory state” depends on the current and voltage history of the device. This is a good example of bioinspired/biomimetic research, since the biological process of synapse formation is mimicked by a physical, electronic device We have previously developed ReRAM as both non-volatile memory (NVM) elements, as well as devices to control the reconfigurability of CMOS circuits.

To date we have developed a full 65nm CMOS/memristor hybrid chip design and fabricate these devices in the SUNY Poly 300mm fabrication facility This work leverages a 65nm node CMOS process design kit (PDK) that has been licensed by the Research Foundation for SUNY for design and fabrication purposes – to which we have added an RRAM module

We have ongoing efforts to characterize the switching mechanism of our devices, to better enable modeling and simulation of memristors in complex circuits We are also investigating methods (both fabrication methods and testing methods) that reduce the stochastic nature of memristor device performance. This will improve reliability of these devices and make the amenable to larger scale integration with complex CMOS circuits for in-memory computing and AI hardware applications

Self-navigating vehicle using RRAM arrays for in-memory computing

Publications

M. Abedin, N. Gong, K. Beckmann, M. Liehr, I. Saraf, O. Van der Straten, T. Ando, N. Cady. Material to system-level benchmarking of CMOS-integrated RRAM with ultra-fast switching for low power on-chip learning. (2023) Nature Scientific Reports. 13, 14963. https://doi.org/10.1038/s41598-023-42214-x

M. Liehr, K. Beckmann, N. Cady. Impact of Switching Variability, Memory Window, and Temperature on Vector Matrix Operations Using 65nm CMOS Integrated Hafnium Dioxide-based ReRAM Devices. (2022) IEEE 31st Microelectronics Design & Test Symposium (MDTS), 2022. https://doi.org/10.1109/MDTS54894.2022.9826924

Acknowledgements

G. Krishnan, Z. Wang, L. Yang, J. Meng, M. Liehr, R.V. Joshi, N.C. Cady, D. Fan, J-S. Seo, Y. Cao. Hybrid RRAM/SRAM in-Memory Computing for Robust DNN Acceleration (2022) IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 41(11): 4241-4252. https://doi.org/10.1109/TCAD.2022.3197516

M. Abedin, M. Liehr, K. Beckmann, J. Hazra, S. Rafiq, N. C. Cady. Inmemory Computation of Error-Correcting Codes Using a Reconfigurable HfOx ReRAM 1T1R Array. (2021) 2021 IEEE International Midwest Symposium on Circuits and Systems (MWSCAS). p. 593-598, doi: https://doi.org/10.1109/MWSCAS47672.2021.9531717

• Air Force Research Laboratory, NSF, Xallent, LLC, IBM- SUNY AI Alliance

Nathaniel Cady

https://www.albany.edu/nanoscale-science-engineering/cady-research-group

Electron micrographs of RRAM devices integrated with CMOS transistors (left) and elemental maps of RRAM device stack (right).

UA-DETRAC Benchmark

• Large scale vehicle detection and tracking evaluation dataset

• http://detrac-db.rit.albany.edu/

AI City Challenge

• Annual CVPR Workshop 2017-2024 and ongoing

• https://www.aicitychallenge.org/

Video Analytics, GE Rail Car Yard Management

Person Detection & Pose Estimation

Acknowledgements

DeepFake Detection

• This work was featured in CNN, The Guardian, Wired, The Register (UK), McClatchy, BBC News and several prominent publications.

DARPA Sematic Forensics Project

• Text-image article consistency analysis

• Image manipulation detection

Multi-Object Tracking

• SMILEtrack is ranked #1 on MOT20 benchmark in paperswithcode.com [AAAI’24]

• https://paperswithcode.com/paper/smiletracksimilarity-learning-for-multiple

AI & Cybersecurity

• Face spoofing detection

• FedTrust: Toward building secure robust and trustworthy moderators for Federated Learning

• DNN Adversarial Attacks and Defense

• Interpretable deep learning

Scientific Machine Learning

• Machine learning inversion approach for determining and inferring effective colloidal interaction potential from scattering

• Image super-resolution for neutron scattering

• We appreciate the following sponsors for grant support or research collaborations:

Prof. Ming-Ching Chang http://www.albany.edu/faculty/mchang2/ Prof. Xin Li https://www.albany.edu/computer-science/faculty/xin-li

[Lipeng Ke, Ming-Ching Chang, Honggang Qi, Siwei Lyu “DetPoseNet: Improving Multi-Person Pose Estimation via Coarse-Pose Filtering ”, IEEE TIP, 2022]

[Ming-Ching Chang, Chi-Huan Tung, Wei-Ren Chen, et al., “A Machine Learning Inversion Approach for Determining Effective Interaction Potential from Scattering”, Communications Physics - Nature, 2022]

Towards DataEfficient Learning via Nonparametric Methods

Project Summary

• Reinforcement Learning (RL) has produced breakthroughs exceeding human proficiency at tasks ranging from playing Atari games to protein structure prediction

• Model-based RL techniques estimate predictive models of the world in order to guide training: what will happen if I do x?

• Parametric model families offer concise model descriptions, but the difficulty of choosing the correct parametrization motivates use of datadriven, nonparametric methods

• In many real-world applications (e.g., healthcare, driving) the experimentation needed to estimate models is costly or unsafe, and only limited, static data sets are available

• For networked settings where cooperative agents may benefit from pooling data, it is often desirable to compress data-driven models given communication bandwidth limitations

• This work aims to develop data-efficient nonparametric techniques for modeling unknown, stochastic nonlinear dynamical systems that can facilitate model-based RL methods, particularly in non-stationary and multi-agent settings

Publications

Highlights

• Analysis of sample complexity for learning transfer operators encoding system dynamics from both i.i.d. and trajectory data, and inaccuracies in dynamical models incurred by estimating sample dynamics from compressed data sets [1]

• Stability analysis for a consensus-based pooling algorithm for multi-agent estimate of dynamics [3]

• A novel, model-based imitation learning (IL) framework exploiting ‘meta’ knowledge of Markovian dynamics which outperforms many state-of-the-art (IL) methods [2]

1. Agrawal, Rishabh, Nathan Dahlin, Rahul Jain, and Ashutosh Nayyar. "Conditional Kernel Imitation Learning for Continuous State Environments." arXiv preprint arXiv:2308.12573 (2023)

2. Hou, Boya, Sina Sanjari, Nathan Dahlin, Subhonmesh Bose, and Umesh Vaidya. "Sparse learning of dynamical systems in RKHS: an operator-theoretic approach." In International Conference on Machine Learning, 2023.

3. Hou, Boya, Sina Sanjari, Nathan Dahlin, and Subhonmesh Bose. "Compressed decentralized learning of conditional mean embedding operators in reproducing kernel hilbert spaces." In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.

Acknowledgements

• Portions of this work supported by the NSF-EPCN-2031570 grant, NSF-CPS-2038775 grant, and C3.ai Digital Transformation Institute

Nathan Dahlin

Initial (top) and refined (bottom) value function estimation of two agents (left/right) using consensus algorithm for OpenAI pendulum swingup task (see [3]).

Application of Machine Learning to Materials

Characterization of Semiconductors

Project Summary

The goal of this project is to determine the crystal phase and orientation of grains of hafnium zirconium oxide (HZO) in thin films used in transistor and memory structures. The films are expected to have a mixture of different crystal phases including a ferroelectric phase.

The data set a map (2D) of electron diffraction patterns which themselves have diffraction information in 2D The diffraction pattern map obtained using scanning transmission electron microscopy is referred to as 4DSTEM data. This project is being done in collaboration with the National Center for Electron Microscopy, The results of a template matching method will be compared to a supervised machine learning approach.

Tokyo Electron is providing metal – HZO –Metal samples and the samples are being electrically characterized for the ferroelectric phase

Publications

Highlights

• Data from collaborator at GaTech analyzed using py4DSTEM software package from NCEM

1. Comparison of template matching and supervised machine learning for characterization of crystal phase and orientation n hafnium – zirconium oxide thing films, A Kordijazi, S Consiglio, S Lompbardo, D Triyoso in progress

2. Ferroelectric phase content in 7 nm Hf(1-x)ZrxO2 thin films determined by X-ray based Methods, V Mukundan, S Consiglio, D H Triyoso, K Tapily, S Schujman, C Mart, T Kämpfe, W Weinreich, J Jordan-Sweet, R D Clark, G J Leusink, and A C Diebold, Physica Status Solidi A: Applications and Materials Science 218, (2021), 2100024

Acknowledgements

• Funding and samples: Steve Consiglio, Sarah Lombardo, Dina Triyoso, Kanda Tapily, and Gert Leusink (TEL)

• Molecular Foundry User Facility funding and collaboration with Colin Ophus (Berkeley Labs NCEM)

• Amir Kordijazi (UNM)

Alain C. Diebold

adiebold@albany.edu

Project Summary Highlights

The recent explosion of artificial intelligence, neuromorphic computing, and machine learning requires new hardware to improve efficiency over software solutions. At the same time, there is an increasing demand for the integration of dissimilar functions (logic, memory, even RF or analog signaling) onto a single platform. While lateral real estate on a chip is limited, threedimensional integration (3DI) enables vertical stacking of functional elements, even chiplets of heterogenous materials The bonding must (1) preserve the metallic character of the interconnects which distribute heat and current; (2) ensure that the surrounding dielectric continues to prevent same; and (3) maintain sufficient robustness to survive subsequent thermal cycling and handling. This program uses dilute copper alloys to address the issues specific to the metal interconnects in 3DI

Publications

Cartoon showing two issues for hybrid bonding: the need for thermal expansion to overcome dishing of the metal, and resistance to plastic deformation and slippage

Alloying Cu with Ag, Co, or other metals can overcome many of the mechanical issues facing metallic bonding for 3DI Shown here for Cu(Ag):

• The Young’s Modulus (stiffness) of the alloy is larger than pure Cu films and can be finely tuned by controlling Ag content

• The Hardness of the alloy is greater than the pure films, as well, also controlled by Ag content

• Taken together, these two mean less plastic deformation & slippage during bonding

• The Coefficient of Thermal Expansion is also controlled by Ag content: the dishing gap can be closed at lower temperatures than if the bond were pure copper

S. Singh and K. Dunn, “Composition-tunable properties of Cu(Ag) alloys for hybrid bonding applications,” Materials 2023, 16, 7481. http://doi.org/10.3390/ma16237481

Acknowledgements

This work was supported in part by the National Science Foundation under award DMR-1808277. Simulation work in support of this program has been conducted by Prof. Bradley Thiel in the department of Nanoscale Science & Engineering, CNSE, University at Albany – SUNY.

Kathleen A. Dunn kdunn1@albany.edu 518-442-8622

https://www.linkedin.com/in/kathleen-dunn-4429185/ https://scholar.google.com/citations?user=EallrNsAAAAJ&hl=en

https://www.albany.edu/nanoscale-science-engineering/faculty/kathleen-dunn

AI Enabled Wireless Connectivity for Indoor Flying Networks

Project Summary

The demand for versatile networks to accommodate the variety of devices and applications is driving the need for NextG wireless communication networks. For example, future applications can impose stringent requirements that will be difficult to satisfy with fixed, pre-deployed infrastructures. The characteristics of unmanned aerial vehicles (UAVs) make them an essential component of the outdoor and indoor NextG infrastructures. For example, UAVs can form indoor-flying networks (IFNs) to support indoor applications. This project introduces the concept of hybrid radio and optical connectivity for IFNs using the WIFi and LiFi technologies, covers the application scenarios of IFNs, discusses the enabling technologies to effectively deploy IFNs, and presents potential research opportunities using artificial intelligence (AI) and machine learning (ML) to enable more efficient communication waveforms and joint communication and sensing.

Publications

Highlights

• The hybrid WiFi-LiFi networks present a significant opportunity for creating lowcost, power-efficient, and reliable IFNs.

• ML-based framework for a unified WiFiLiFi transceiver can improve the performance and reduce the transceiver complexity.

• The performance and complexity of deep neural networks (DNNs) and convolutional neural networks (CNNs) based transceivers are investigated.

• CNNs outperform traditional DNNs in WiFi channels. In contrast, traditional DNNs perform better than CNNs in LiFi channels.

• Ahmad, R., Elgala, H., Almajali, S., Salameh, H.B. and Ayyash, M., 2023. Unified physical layer learning framework towards VLC-enabled 6G-indoor-flying-networks. IEEE Internet of Things Journal.

• Insert Ahmad, R., Ayyash, M., Salameh, H.B., El-Khazali, R. and Elgala, H., 2023. Indoor Flying Networks for 6G: Concepts, Challenges, Enabling Technologies, and Opportunities. IEEE Communications Magazine.

Acknowledgements

• Collaborating Universities: Chicago State University, USA , Al-Ain University and Khalifa University, UAE.

• Funding sources: This work is supported by the ASPIRE Award for Research Excellence Program 2020 (Abu Dhabi, UAE) under Grant AARE20-161

https://www.albany.edu/ece/faculty/hany-elgala

Zero Energy Nano (ZEN)

• Seamless access to data archive

• Easy, flexible trends and graphs

• View metrics across assets

ZEN-NFX Data Warehouse (Digital Sandbox)

NanoFab eXtension (NFX)

ZEN designed as “Living Laboratory” with 8,000 data “tags” associated with 749 systems used to operate as “mission critical” with 30,000 sq.ft. NYS Excelsior Cloud data center integrating active & passive energy efficiency measures to achieve net zero operation. Expanding into NFX and Central Utility Building (CUB).

Building Management System (BMS) “Loop Monitoring & Control”

Performance Mang’t. System (PMS) “Advanced Analytical Tools”

• Python Library to Solve problems quickly

• Machine Learning

• Artificial Intelligence

Publications

Process-grade Monitoring & Controls

“Academia Driving Deployment of Digital Twins for Zero Energy Market Adoption”, High Profile Focus: Schools & Institutions, March 2023.

Building Analysis System (BAS) “Dashboard Analytics”

Continuous Commissioning Platform

• Hundreds of analytics

• Fleet View

• Fault Causes

Building Energy Model

“Digital Twin”

Acknowledgements

• NYS Energy Research & Development Authority

• U.S. Dept. of Commerce (EDA)

• NYS Empire State Development (ESD)

• Japan New Energy & Tech. Dev. Office (NEDO)

Michael Fancher

Email: mfancher@albany.edu

https://www.catn2.org/

https://www.amperformance.org/

An optimization-based decision support system for cost-effectively implementing best management practices

Project Summary

• Hypoxia & harmful algal blooms (HABs), primarily caused by excess nonpoint source (NPS) nitrogen and phosphorus loadings in U S waters, have caused significant economic losses and health issues

• Reducing NPS nutrient losses from agricultural areas by agricultural best management practices (BMPs) is the key to solving the issues.

• This project addresses the challenges of creating cost-effective and sustainable agricultural BMP implementation strategies by developing an optimization-based decision support system (SWAT-BMP-OPT) incorporating improved AMALGAM (A Multi-ALgorithm Genetically Adaptive Multi-objective method, which is a popular optimization method using artificial intelligence techniques), enhanced SWAT (Soil and Water Assessment Tool), and modified BMP-COST (BMP Cost Evaluation Tool)

Highlights

• Create and enhance SWAT, BMP-COST, and SWAT-BMP-OPT to optimize agricultural BMP implementation strategies that minimize nutrient loadings at minimum costs

• Apply the improved SWAT, BMP-COST, and SWAT-BMP-OPT, starting with agricultural BMPs already implemented in the case study area, to answer various questions.

Publications

• Liu et al., 2019. Evaluating efficiencies and cost-effectiveness of best management practices in improving agricultural water quality using integrated SWAT and Cost Evaluation Tool. Journal of Hydrology.

DOI:10.1016/j.jhydrol.2019.123965

• Liu et al., 2019. A SWAT-based optimization tool for obtaining cost-effective strategies for agricultural conservation practice implementation at watershed scales. Science of the Total Environment.

DOI:10.1016/j.scitotenv.2019.07.175

Acknowledgements

• Funding source: United States Department of Agriculture-National Institute of Food and Agriculture (USDA-NIFA) grant 2023-67021-40619

Yaoze Liu

yliu46@albany.edu

Website: https://www.albany.edu/environmental-sustainable-engineering/faculty/yaoze-liu

Theoretical foundations for learning in nonclassical settings

Project Summary

Dynamic processes on complex networks of interconnected entities abound as models of physical, biological, engineered, social, and economic systems Learning flexible, human-interpretable statistical models of these processes from data is a vital prerequisite for their prediction and control. Designing machine learning models and algorithms, even in a non-networked setting, is not fully understood mathematically. The networked nature of the processes brings additional challenges for model accuracy, human interpretability, and efficiency This project seeks a new mathematical framework, efficient algorithms, and theoretical guarantees for learning data-driven models of evolving processes on complex networks. It aims also to provide rigorous techniques for validation and interpretation of learned models. These advances should result in improved model validation, accuracy, efficiency, and interpretability for diverse, societally impactful application domains such as epidemiology, misinformation spread on social networks, electrical power networks, supply chain networks, nutrient diffusion processes in agriculture, and quantum learning scenarios.

Highlights

• Developed a multi-temporal-scale statistical model for network spreading processes, along with provably efficient and accurate model parameter estimation algorithms.

• Precisely characterized the conditions for learnability of quantum measurement models for quantum supervised learning problems.

• Initiated a theory of the expressive power of graph machine learning models, a key step toward devising provably efficient and trustworthy models.

• Probed the limitations of state of the art methods for graph summarization, a key tool used for efficient machine learning on graphs.

Publications

• Arun Padakandla and Abram Magner, “Pac learning of quantum measurement classes : Sample complexity bounds and universal consistency,” in Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, Eds. 28–30 Mar 2022, vol. 151 of Proceedings of Machine Learning Research, pp. 11305–11319, PMLR.

• Abram Magner and Carolyn Kaminski and Petko Bogdanov, “Fast and accurate spreading process temporal scale estimation,” in Transactions on Machine Learning Research, 2022.

• Sepideh Neshatfar, Abram Magner, and Salimeh Yasaei Sekeh, “Promise and limitations of supervised optimal transport-based graph summarization via information theoretic measures,” 2023.

Acknowledgements

• NSF CCF # 2338855: CAREER: Theoretical Foundations for Learning Network Dynamics

• NSF CCF # 2212327: Collaborative Research: CIF: Medium: Foundations of Robust Deep Learning via Data Geometry and Dyadic Structure

Abram Magner

https://www.albany.edu/faculty/amagner

Black-Box Learning of Web Authorization Policies

Project Summary

Web applications have become the de facto way to access services and functionalities. Ensuring that different users of web applications are only allowed to access what they are supposed to implementing correct authorization is vital. Unfortunately, broken access control and authorization consistently rank among the top web application vulnerabilities. In fact, many web applications struggle to provide an accurate specification of their enforced authorization policies due to challenges like code complexity and fastpaced development. Neither end users nor even developers of these applications can easily reason about data protection in this environment. To address this problem, this research project introduces a novel paradigm for automated learning of web application authorization policies that significantly improves ensuring the security and privacy of web applications A key characteristic of this research is treating web applications as black boxes, i.e., learning authorizations by interacting with and observing them The black-box approach allows us to abstract away internal complexities and focus instead on what really matters: learning the policies enforced on users as they access application data.

Publications

Highlights

• We have developed a first-of-its-kind framework for learning web authorization policies using automated interaction and inspection of applications’ web requests/responses. We capture the learned policies formally as deterministic finite automata.

• Our current experimental results on 10 applications from different domains demonstrate promising accuracy for learning authorization rules on finegrained data captured in those applications.

• We are currently conducting research to further optimize our learning architecture, capture more fine-grained policies, and develop black-box security analysis techniques based on our learning framework.

• P. Iyer and A. Masoumzadeh, “Towards Automated Learning of Access Control Policies Enforced by Web Applications,” in Proceedings of the 28th ACM Symposium on Access Control Models and Technologies, 2023, pp. 163–168.

• P. Iyer and A. Masoumzadeh, “Learning Relationship-Based Access Control Policies from Black-Box Systems,” ACM Transactions on Privacy and Security, vol. 25, no. 3, pp. 22:1–22:36, Aug. 2022.

• P. Iyer and A. Masoumzadeh, “Effective Evaluation of Relationship-Based Access Control Policy Mining,” in Proceedings of the 27th ACM Symposium on Access Control Models and Technologies, 2022, pp. 127–138.

Acknowledgements

This project is sponsored by the National Science Foundation through NSF CAREER award #2047623.

Amir Masoumzadeh

Associate Professor, Computer Science

amasoumzadeh@albany.edu

https://www.cs.albany.edu/~amir/

Heterostructure

Phase Change Memory for Multilevel Programming

Project Summary

• Recent reports on heterostructure phase change memory (PCM) cells, in particular with Van der Waals (VdW) gaps, have demonstrated their beneficial performance characteristics, including lower variability and drift, lower switching energy, and as a result better suitability for multilevel operation in the AI-inspired computer architectures.

• Concurrently, our studies of the binary Al-Sb system with phase separation and therefore short-range heterogeneity shows excellent PCM performance characteristics.

• In this project, we engineer the PCM cells combining the low thermal boundary conductance associated with VdW bonding with reduced stochasticity of Sb-rich phase crystallization by inhomogeneous nucleation at a lowenergy interface with TiTe2 template

• The materials and technology components of the research are focused on mechanisms, microstructure, chemistry, and bonding changes due to switching, the kinetics of phase transformations, and build-up strain

Publications

Highlights

• High amorphous phase stability of Al-Sb and Ga-Sb (vs. Ge2Sb2Te5)

• Crystallization times are ~30% shorter than for Sb-Te alloys

• Possible to crystallize/amorphize just low-meltingtemperature Sb-rich phase  reduce energy for switching

• Reversible control of alloying of the Sb-rich phase novel mechanism for multilevel/analog resistive memory

(a) TEM micrograph of a 20 nm thick Al0.4Sb0.6 PCM film on TiN heater. (b) Switching of the cell.

(a) Programming resistance curves of 20 nm thick Al0.4Sb0.6 PCM cell for different pulse widths. (b) Low drift of the cell.

• R. Ume, et al. “Crystallization Properties of Al-Sb Alloys for Phase Change Memory Applications.” ECS J. Solid State Sci. Technol., 10, 075008 (2021).

• H. Gong, et al. "Bilayer Ga-Sb Phase Change Memory with Intermediate Resistance State." Device Research Conference 2021.

• R. Ume, et al. "Electrical and structural properties of binary Ga–Sb phase change memory alloys," J. Appl. Phys. 132, 035103, 2022.

Acknowledgments

• The SUNY-IBM AI Research Alliance

• Semiconductor Research Corporation

Serge Oktyabrsky

soktyabrsky@albany.edu

Heterostructure Phase Change Memory for Multilevel Programming

Project Summary

Joint UAlbany and UBuffalo team focus:

(a) explore the properties and potential benefits of the adaptive IR spectral sensitivity selection and associated image processing in changing environment, and

(b) design and build a unique AI object recognition system with integrated IR dual-band adaptive sensors (both MWIR and LWIR bands) based on asymmetrically doped double-quantumwell structures.

Objectives:

• Build, test, and explore the properties and potential benefits of an integrated AI system with sensor adaptation capability.

• Develop adaptive IR image acquisition and processing capabilities to maximize object recognition performance

• Develop and demonstrate IR wide band adaptive imaging sensor based on asymmetrically doped doublequantum-well structures covering both 3-5 (MWIR) and 8-14 µm (LWIR) spectral ranges

• Demonstrate the object recognition performance improvement of the thermal infrared sensor based on asymmetrically doped dual quantum wells.

Publications

Highlights

• The distinguishing characteristic of this sensor is the tunability of the photosensitive spectrum depending on the voltage applied to the sensor.

• To take advantage of the increased capabilities of the sensor, we propose to develop image processing algorithms optimized for the best object recognition performance with thermal features producing image contrast in MWIR and LWIR spectral regions.

• IR sensor based on asymmetrically doped dual quantum wells is the best candidate for the AI-driven imaging applications in the IR.

Operation of AI -Driven Adaptive Sensor

(a)

(b)

(a) Spectral response of photodetector tunable between LWIR and MWIR regions with applied bias. (b) Schematics of dual-band adaptive IR imager

• S. Tulyakov, V. Mitin, G. Biswal, M. Yakimov, V. Tokranov, and K. Sablon, "Object Recognition for Multiband Thermal Infrared Sensing," in 2023 IEEE Long Island Systems, Applications and Technology Conference (LISAT), 2023, pp. 1-6.

• G. R. Biswal, M. Yakimov, V. Tokranov, K. Sablon, S. Tulyakov, V. Mitin, and S. Oktyabrsky, "Development of voltage-tunable IR photodetector," in Infrared Sensors, Devices, and Applications XIII, 2023, pp. 78-83.

Acknowledgments

• U.S. Army Research Office

• U.S. Airforce Office of Scientific Research / Air Force Research Lab

Serge Oktyabrsky, soktyabrsky@albany.edu

Collaborator: Vladimir Mitin, vmitin@buffalo.edu

SIENNA- A Machine Learning Medical Imaging

AGI Companion Diagnostic

Project Summary

The Paluh laboratory is actively engaged in cutting edge AI neurotechnologies to advance Machine Learning applications in the clinical setting. We focus on smaller models with reduced parameters to generate high accuracy coupled with high F1 scores and AUROC values. Deep learning models are notoriously challenged by shortcut learning, leading to Inability to effectively generalize these models across differing patient datasets or to learn effectively. SIENNA is designed to address such challenges. SIENNA is a state-of-the-art multi-class classifer for brain tumor diagnostics and being expanded for subtyping correlations with tumor genetics and tumor primary origin diagnostics.

Highlights:

• SIENNA AGI platform

• Bypass shortcut learning

• DICOM T2 FLAIR image analysis

• Hyperparameter tuning

• Adversarial training

• Grad-CAM analysis

• Minimized false positive/false negatives

• AGI for biomedical diagnostic MRI

• Clinical training accelerator

Publications:

S.S. Kurup, R.S. Rajeev, J.Pilitsis, A. Chatterjee, A. Mukherjee, & J.L. Paluh (2023) SIENNA: Lightweight generalizable machine learning platform for brain tumor diagnostics. Submitted Nature Machine Intelligence

B. Sai T. Naidu, S. Biswas, R. Chatterjee, S. Mandal, S. Pratihar, A. Chatterjee, A. Raha, A. Mukherjee and J.L. Paluh (2022) SCENIC: An area and energy-efficient CNN-based hardware accelerator for discernable classification of brain pathologies using MRI. International Conference on Embedded Systems (VLSI Design Conference).

Acknowledgments:

• SUNY RF Technology Accelerator Fund (TAF) Award

• Dr. Julie Pilitsis MD, PhD MBA Prof & Chair Dept Neurosurgery, University of Arizona

• ITrakNeuro Inc Startup (JL Paluh, A Mukherjee, A Chatterjee, R Chatterjee)

Janet L. Paluh, Ph.D jpaluh@albany.edu

Dept Nanoscale Science & Engineering

Website: https://www.albany.edu/nanoscale-science-engineering/faculty/janet-paluh Research: https://www.albany.edu/nanoscale-science-engineering/paluh-research-group Startup: ITrakNeuro.com Biomedical AI Diagnostics, Training & Communications

Machine Learning for Wireless Signals

Project Summary

• Collaborative Interference Cancellation for Radio Astronomy

• Neural Network Model for Communication in sub-Terahertz frequencies

• WISE: Waveform Independent Signal Embedding for Covert Communication

• Detecting UNclassified Electromagnetic Signals (DUNES) for Secure Wireless Communication

Highlights

• Developed neural network models with domain specific knowledge for faster convergence

• Applications involve improved communication in new higher frequencies, interference cancellation for radio astronomy, and secured signals.

Publications

• Xue Wei and Dola Saha, WISE: Waveform Independent Signal Embedding for Covert Communication, accepted in IEEE Transactions on Machine Learning in Communications and Networking (TMLCN), 2024.

• Shuvam Chakraborty, Dola Saha, Aveek Dutta and Gregory Hellbourg, LOCI: Learning Low Overhead Collaborative Interference Cancellation for Radio Astronomy, in IEEE ICC 2023.

• Shuvam Chakraborty, Dola Saha and Ngwe Thawdar, Communication Knowledge Aided Neural Network for OFDM Receiver in Terahertz Band, in IEEE International Conference on Communication 2022.

Acknowledgements

• Key collaborators: Prof. Aveek Dutta (UAlbany), Prof. Hany Elgala (UAlbany), Dr. Gregory Hellbourg (Caltech), Dr. Randall Reese (INL), Dr. Anna Quach (INL), Dr. Nicholas Kaminski (INL)

• Funding sources: National Science Foundation (NSF), Air Force Research Laboratory (AFRL), Air Force Office of Scientific Research (AFOSR), Idaho National Laboratory (INL), Missile Defense Agency (MDA), National Aeronautics and Space Administration (NASA)

https://www.albany.edu/faculty/dsaha/

Dola Saha

Project Summary

Automatic personalized recommendation tools are crucial in many domains, including health, online education, and retail. These tools provide personalized guidance to users and suggest the most appropriate items or actions to them under uncertain conditions.

Ideally, these recommendations should suggest items of different types (e.g., to study the most effective learning material type, whether it is a video lecture, book chapter, or practice problem in education), at the right time, to efficiently accomplish the individual’s multiple long-term goals. Current solutions, focus on suggesting one type of item, ignoring the importance of recommendation timing, and only satisfying one goal for all users. This project will research a new generation of recommender systems towards achieving users’ long-term goals by automatically detecting and balancing between their different, potentially conflicting goals and detecting the best timing to provide suggestions. It contributes to the offline policy evaluation in recommender systems, time-aware reinforcement learning, multi-objective optimization, and policy optimization with long-term rewards.

Publications

Highlights

• We developed the first personalized deep learning model to track student knowledge over long learning trajectories and predict their performance.

• We developed a model for effective evaluation of sequential treatment vs. control policy experiments without real-world testing of the treatment policy (using offline data).

• We developed a personalized model of user stress, activities, and app engagement sequences to predict future user stress according to their historical activities and find which activities are associated with lower stress levels for what kind of users.

• C. Wang, and S. Sahebi. “Continuous Personalized Knowledge Tracing: Modeling Long-Term Learning in Online Environments.” Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM). 2023.

• S. Zhao, and S. Sahebi, “Graph-Enhanced Multi-Activity Knowledge Tracing”, Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD). 2023.

• C. Wang, S. Zhao, and S. Sahebi, “Transition-Aware Multi-Activity Knowledge Tracing." 2022 IEEE International Conference on Big Data (Big Data). 2022

• C. Wang, S. Sahebi, and P. Brusilovsky “Proximity-based educational recommendations: A multi-objective framework”, The 2nd Workshop on Multi-Objective Recommender Systems (MORS’22), 2022.

• C. Wang, S. Sahebi, and H. Torkamaan “STRETCH: Stress and Behavior Modeling with Tensor Decomposition of Heterogeneous Data”, The 2021 IEEE/WIC/ACM International Joint Conference On Web Intelligence And Intelligent Agent Technology (WI-IAT), 2021.

• C. Wang, S. Sahebi, and P. Brusilovsky “MOCHI: an Offline Evaluation Framework for Educational Recommendations”, The Workshop on Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES’21), 2021.

Acknowledgements

• This material is based upon work supported by the National Science Foundation under Grant No. 204750

Sherry Sahebi

Associate Professor, Computer Science ssahebi@albnay.edu

https://www.cs.albany.edu/~sherry/ PersAI Lab: https://persai-lab.github.io/

Detecting and Predicting Procrastination in Online and Social Learning

Project Summary

Online education is increasingly available and trusted by both employers and students. However, procrastination remains a key challenge to time management and success in such education context. This project develops computational algorithms to model individual students’ procrastination behaviors, study them along psychological and educational procrastination theories, predict when a student will return to study, and estimate the student’s performance in the learning according to learners’ prior studying behavior. It contributes to the point process modeling of temporal and noisy data with missing values, neural recurrent point processes, marked point processes, and granger causal modeling of temporal data The findings of this project can be used to enhance students’ learning and applied to related areas such as workforce development.

Highlights

• We simultaneously predict the timing and outcome of an event and model the relationship between different event types and times by developing a recurrent neural marked point process model.

• We predict when a student would start working on an assignment by developing the first collaborative Hawkes process model that can estimate event intensities for a user without observed data.

• We find groups of students according to their procrastination behavior patterns and find the association between these behavior groups and student performance.

Publications

• S. Sahebi, M. Yao, S. Zhao, and R. Feyzi Behnagh.“MoMENt: Marked Point Processes with Memory-Enhanced Neural Networks for User Activity Modeling” In ACM Transactions on Knowledge Discovery in Data. 2024

• S. Zhao, S. Sahebi, and R. Feyzi Behnagh. “Curb Your Procrastination: A Study of Academic Procrastination Behaviors vs. A Planning and Time Management App.” In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization. 2023.

• M. Yao, S. Zhao, S. Sahebi, and R. Feyzi Behnagh. “Relaxed clustered hawkes process for student procrastination modeling in MOOCs.” In Proceedings of the AAAI Conference on Artificial Intelligence. 2021.

• M. Yao, S. Zhao, S. Sahebi, and R. Feyzi Behnagh. “Stimuli-sensitive Hawkes processes for personalized student procrastination modeling.” Proceedings of the Web Conference. 2021.

• M. Yao, S. Sahebi, R. Feyzi Behnagh, S. Brusali, and S. Zhao. “Temporal processes associating with procrastination dynamics”. In Artificial Intelligence in Education. 2021.

• M. Yao, S. Sahebi, and R. Feyzi Behnagh. “Analyzing student procrastination in MOOCs: a multivariate Hawkes approach.” In Proceedings of the 13th Conference on Educational Data Mining. 2020.

Acknowledgements

• This material is based upon work supported by the National Science Foundation under Grant No. 1917949

Sherry Sahebi

Associate Professor, Computer Science

ssahebi@albnay.edu

https://www.cs.albany.edu/~sherry/ PersAI Lab: https://persai-lab.github.io/

Functional and Integrative Tissue Generation Center (FITGC)

Project Summary

Functional and Integrative Tissue Generation Center (FITGC) aims to promote convergent research, education and workforce training, and technology innovation and translation for widelyaccepted adoption of next generation tissue engineering.

• Research on functional cell-biomaterial interactions and multicellular interactions for directed cell regulation based on biomaterial science and cell biology/physiology;

• Development of enabling technologies for engineering cell organization and tissue function employing real-time biosensing to promote longevity and functional outcomes while enabling biomanufacturing in a scalable and reproducible manner;

• Translation of innovation in integration to generate reproducible, scalable, functional cell/tissue/organoid products for biological studies, drug testing, cell therapy and regenerative medicine.

Publications

Highlights

• Bioengineering Stromal Cell Delivery and Organoid Culture for Salivary Gland (SG) Regeneration

- Nanofiber scaffolds for SG regeneration

- AI-assisted bioengineering of SG organoids

- Raman & AI-enabled detection of organoids

• Bioengineering Ocular Outflow Physiology for High-throughput Drug and Gene Screening

• Jorgensen M, Ramesh P, Toro M, Evans E, Moskwa N, Zhang X, Sharfstein ST, Larsen M, Xie Y. Alginate hydrogel microtubes for salivary gland cell organization and cavitation. Bioengineering. 2022;9(1):38. doi: 10.3390/bioengineering9010038.

• Ramesh P, Moskwa N, Hanchon Z, Koplas A, Nelson DA, Mills KL, Castracane J, Larsen M, Sharfstein ST, Xie Y. Engineering cryoelectrospun elastin-alginate scaffolds to serve as stromal extracellular matrices. Biofabrication 2022;14(3): 035010. doi: 10.1088/1758-5090/ac6b34.

Acknowledgements

• Susan Sharfstein, FITGC Co-director, Xulang Zhang, FITGC Manager, and Ross Goodman

• Melinda Larsen (salivary tissue regeneration), Alexander Khmaladze (Raman spectroscopy and 3D imaging), Yunlong Feng (machine learning), Petko Bogdanov (data mining), John Danias (glaucoma) & Yiqin Du (stem cell)

• The New York State Center for Advanced Technology in Nanomaterials and Nanoelectronics (NYS CATN2), FuzeHub Manufacturing Grant, SUNY Poly Seed Grant, Xheme (Sharfstein), NIIMBL PC4.1-146 (Sharfstein), NSF ECCS (Sharfstein), & NIH NIDCR R01DE027953 (Larsen)

Yubing Xie, Ph.D.

Professor of Nanoscale Science and Engineering

FITGC Director

Email: yxie@albany.edu

https://www.albany.edu/nanoscale-scienceengineering/faculty/yubing-xie