CyberByte - Fall 2023

Page 1

FA L L 2 0 2 3

FA L L 2 0 2 3

Well to our class of 2023 Ph.D. graduates, including Dr. Gary Mac (bottom left) and Dr. Prashant Rajput (bottom right). Meanwhile, we look forward to the coming semester and the new initiatives of the NYU/KAIST partnership. Top photo was taken at a planning workshop held at KAIST on May 15-16.

Celebrating what has passed….Planning for what is to come As the seasons turn, we say a fond farewell to our class of 2023 Ph.D. graduates, including Dr. Gary Mac and Dr. Prashant Rajput. Meanwhile, we look forward to the coming semester and the new initiatives of the NYU/KAIST partnership. Photo on the top left above was taken at a planning workshop held at KAIST on May 15-16.

CENTER FOR

CYBERSECURITY

1


FA L L 2 0 2 3

A NOTE FROM THE EDITORIN-CHIEF Power generation plants, municipal water plants, and fuel pipelines are just a few of the elements that make up the critical infrastructure that just about every country, city or village in the world relies upon. When these large-scale mechanical systems added computer monitoring and control, such operations were able to run faster and more efficiently. Unfortunately, this change also opened a popular new attack surface, particularly for well-organized and funded nation-state hackers. While the Colonial Pipeline hack in the spring of 2021 perhaps attracted the most attention, it was far from an anomaly. The FBI’s Internet Crime Report for 2022 noted a total of 2,385 reported ransomware attacks in 2022, of which 870 targeted critical infrastructure organizations. (You can read the report at https://www.ic3.gov/Media/PDF/AnnualReport/2022_ IC3Report.pdf). As the attackers taking on these systems get better at their jobs, the time has come for cyber professionals from a variety of backgrounds to roll up their sleeves and up the ante on defensive strategies. Here at the Center for Cybersecurity— both in Brooklyn and in Abu Dhabi—a core of researchers are engaged in designing better protection for physical and communications infrastructures. Such strategies recognize that, in the words of our Faculty Profile subject, NYU Abu Dhabi associate professor Michail Maniatakos, rarely can an attack “be efficiently created targeting an isolated abstraction layer.” Thus, most of these initiatives, including Maniatakos’ efforts, are interdisciplinary in approach. For Maniatakos, an “interdisciplinary” approach involves crafting solutions in which “hardware techniques need software support and vice versa.” On a larger scale, Tandon’s multifaceted approach for defending critical systems is also displayed in the work of six researchers at the school who received a grant from the National Science Foundation to “make current and future wireless infrastructure, software and hardware systems more resilient to flaws, accidents, subterfuge and hacks.” In this issue you can read more about this project, which counts CCS faculty members Drs. Siddharth Garg and Ramesh Karri as part of the research team. Lastly, we salute our Class of ‘23 Ph.D. graduates, check in with recent alumnus Dr. Zahra Ghodsi, who is now an assistant professor at Purdue, and catch up on the many events, programs, and awards presentations involving CCS faculty and students over a busy spring and summer semester. Enjoy.

Dr. Quanyan Zhu, Editor in Chief

2


FA L L 2 0 2 3

IN THIS ISSUE A NOTE FROM THE EDITOR-IN-CHIEF

2

CELEBRATING OUR 2023 PH.D GRADUATES

4

RESEARCH FOCUS: PROTECTING THE CRITICAL SYSTEMS OF MODERN LIFE

5

FACULTY PROFILE: MICHAIL MANIATAKOS

8

PH.D. PROFILE: PRASANT ADHIKARI

10

ALUMNI PROFILE: ZAHRA GHODSI

12

CCS NEWS

14

EVENTS

17

AWARDS AND HONORS

20

STAFF

Editor in Chief

Editorial Copy Writer

Quanyan Zhu

Lois Anne DeLong

3


FA L L 2 0 2 3

CELEBRATING OUR CCS PH.D. GRADUATES

YUNHAN HUANG (Advisor: Dr, Quanyan Zhu) Ph.D. Electrical Engineering Yunhan, who completed his degree in the summer of

The NYU Center for Cybersecurity is proud to recognize its 2023

2022, focused his research

Ph.D. graduates. This year’s newly minted graduates marked their

efforts around information

departure at a ceremony held May 18, 2023 at the Barclays Center.

security and optimizing

Congratulations to the Center’s newest doctors.

information utilization in sequential decisionmaking processes. More precisely, his investigations

YINAN HU

tackled two compelling

(Advisor: Dr, Quanyan Zhu)

inquiries: first, the strategic acquisition and cost-effective utilization

Ph.D. Electrical Engineering

of information in sequential decision-making, and secondly, the potential impact of manipulated information or data on the efficacy

Yinan’s research work

of reinforcement learning algorithms and their underlying systems.

at Tandon included

Over the course of four years, Yunhan authored and co-authored

a study of quantum

a total of 14 academic papers dedicated to finding resolutions to

detection frameworks

these dual challenges. He has also actively participated in IEEE

under adversarial

conferences over the years, including serving as chair of the 2020

environments, a topic

IEEE International Workshop on Information Forensics and Security

which has applications in

in New York City, as general co-chair of the HPCC Workshop

cybersecurity and target

on Artificial Intelligence Empowered Efficient and Secure 6G

detection. His dissertation

Networking and Communications in 2021, and as publication chair

explores “Advanced and

of the 20th International Conference on Embedded and Ubiquitous

Contemporary Applications of Classical and Quantum Detection

Computing in 2022. He is now working at Amazon as an applied

Theories in Strategic Environments.” Most recently, he presented at

scientist who promotes the efficient use of information/context in

the 61st IEEE Conference of Decision and Control (CDC), sharing

Large Language Models.

a holistic framework for proactive detection that combats evasive attacks in network security.

PRASHANT HARI NARAYAN RAJPUT

GARY MAC

(Advisor; Dr. Michail

(Advisor: Dr. Nikhil Gupta)

Maniatakos)

Ph.D. Mechanical and

Ph.D. Computer Science

Aerospace Engineering

Prashant was a Global Ph.D.

Gary’s interdisciplinary

Fellow at NYU Tandon,

research focused on

where he contributed to

cybersecurity risks in

two different research

digital manufacturing

projects revolving around

of cyber- physical

malware detection. His

systems. He developed

dissertation research on

safe and trustworthy

“Hardware-Assisted Non-

countermeasures to protect

Intrusive Security Controls for Modern Industrial Control Systems”

products against intellectual

began with a study of the impact of process-aware cyberattacks on

property theft and counterfeit production. Gary also served as the

critical infrastructure, such as thermal desalination plants. Using the

global competition coordinator for the annual CSAW Hack3D event,

data gathered in the study, he built non-intrusive security controls

which uses crowdsourcing a diverse group of students as a method

for modern ICS devices at both the user and kernel levels. This led

to evaluate the robustness of novel digital manufacturing security

him to create ICSPatch, a tool that “identifies, localizes, and hot

strategies. Over the past five years, the Hack3D competition has

patches vulnerabilities in the control application, the logic that

engaged more than 1,800 students worldwide, and has inspired a

controls the physical industrial process.” Since graduating, Prashant

partnership with ASME to co-host a student hackathon.

has started work as a developer support engineer at InterSystems.

4


FA L L 2 0 2 3

RESEARCH FOCUS:

PROTECTING THE CRITICAL SYSTEMS OF MODERN LIFE Creating effective cyber defenses for the emerging threats against infrastructure carries with it a number of special challenges. For example, in a set of presentations at CSAW’22, Ali Naqvi, a manager with Consolidated Edison’s Cybersecurity Operations Center, pointed to “protocols that may or may not be standardized, a limited number of defensive tools, and increasingly stringent regulations.” And, Abhishek Ramchandran of Siemens noted that obsolete components in a facility can be hacked and used as a way to leverage access into critical infrastructure systems. At the NYU Center for Cybersecurity, faculty members and students are identifying and adapting to these distinctions, and creating defense strategies that respond to the specific needs of different industry segments. A few recent examples are presented below.

WRITING THE BOOK ON INDUSTRY-SPECIFIC RISKS TO CRITICAL INFRASTRUCTURE Over the past few years, Dr. Quanyan Zhu, an associate professor of electrical engineering, has published a series of textbooks that elaborate on the best defensive strategies against threats to specific industries. Given the increasing usage of robotics in a number of industrial segments—one report noted an average of 141 robots per every 10,000 employees—it’s not surprising he has turned his attention to potential future attacks in this field (see https://ifr.org/img/worldrobotics/Executive_Summary_WR_Industrial_ Robots_2022.pdf). Zhu, along with colleagues from Austria, released a book entitled Cybersecurity in Robotics: Challenges, Quantitative Modeling, and Practice. Declaring its purpose in the introduction, the authors emphasize “the inclusion of security in robotics from the earliest design phases onward and with a special focus on the cost-benefit tradeoff that can otherwise be an inhibitor for the fast development of affordable systems.” The book also advocates for “quantitative methods of security management and design, covering vulnerability scoring systems tailored to robotic systems, and accounting for the highly distributed nature of robots as an interplay of potentially very many components.” Like much of Zhu’s work, the book incorporates the use of game theory as part of a “quantitative approach to model-based security.” Zhu also has examined threats to another facet of industrial infrastructure: control systems. In Security and Resilience of Control Systems: Theory and Applications, a book he co-edited with Hideaki Ishii of the Tokyo Institute of Technology, Zhu acknowledges that the key to defending these essential components of modern industry is to base approaches in “the knowledge and models of the physical systems, rather than an attempt to reinvigorate conventional IT-based security measures.” The book also warns that attacks can come in different guises, which “may not be detected unless the security measures exploit the physical models.”

5


FA L L 2 0 2 3

Biochip Defense Countermeasures: From the article “Material-Level Countermeasures for Securing Microfluidic Biochips,” the cover story of the August 2023 issue of Lab on a Chip (photo at right). The illustration depicts two countermeasures to address material-level attacks on microfluidic biochips. On the left, an ML-based material authentication scheme where a simple punch test (F), coupled with a load cell, can authenticate whether the material’s composition is correct. On the right, a laser shining UV light (λ) on the dye-doped microvalve provides a spectral signature with peaks that dynamically change based on the mechanical deformation of the valve..

DEFENDING BIOCHIPS According to the FBI’s Internet Crime Report for 2022, 210 of a total of 2,385 reported

paper documents how “the mechanical

ransomware attacks that year were directed against hospitals and public health care

performance of MBs could degrade

organizations. One particular vulnerability in this arena lies in the use of microfluidic

if reactive or deteriorating chemicals

biochips (MBs), or devices that “miniaturize and integrate various laboratory processes

were intentionally introduced during

onto a single platform.” This summer, Navajit Singh Baban, currently a postdoctoral

fabrication.” Baban and his team

associate at the Center for Cyber Security at NYU Abu Dhabi, tackled this issue during a

developed a “machine learning-based

three week research experience at the NYU campus in Brooklyn.

method capable of detecting doped

Baban was one of seven post-doctoral researchers selected for this year’s Abu Dhabi

combined with a force sensor, an

Collaborative Grant Program. From July 9 to 29, these individuals worked alongside

approach that achieved an accuracy

outstanding researchers in Brooklyn and Manhattan. For Baban, this meant collaborating

rate of over 99% in identifying material-

with Drs. Ramesh Karri and Nikhil Gupta on a project entitled “Providing Secured and

level anomalies.” The research team also

Trustworthy Cyberphysical Systems for Microfluidic-based Biological Computing Devices.”

developed “a dynamic watermarking

As Baban explains in a report he drafted for CyberByte, MBs are “finding commercial

scheme using a mechanoresponsive

applications in a growing number of fields, including biomedical research and diagnostics,

dye,” that enabled “a spectral signature

because they can not only conserve resources but also accelerate research timelines.”

change under ultraviolet laser exposure.”

Attacks on these devices, “imperil patient well-being, erode trust, squander resources,

This watermark makes MBs resistant

and trigger economic consequences,” and therefore there is a need for “swift, coordinated

to reverse engineering attempts. The

materials using a simple punch test

countermeasures” to “maintain biochips’ reliability, confidentiality, and trustworthiness.”

research initiative was funded by the

During his time in New York, Baban set out to complete “a comprehensive exploration

Karri and Krishnendu Chakrabarty, Fulton

of potential vulnerabilities and attack vectors associated with MBs.” The first step was

Professor of Microelectronics from

to conduct benchtop experiments to empirically demonstrate these potential attacks.

Arizona State University, serving as

With the knowledge gained from these experiments, the project could then focus on

principal investigators on the grant. Other

“devising effective countermeasures.” These defensive strategies include developing a

collaborators included researchers from

suite of resources capable of safeguarding MBs from threats like “manipulation of bioassay

Duke University in Durham, NC, and the

outcomes, tampering with biomolecular protocols, denial-of-service disruptions, intellectual

Indian Institute of Technology in Guwahati,

property theft, and biopiracy, among other perils.”

India. Credit also goes to Ross Fleming,

National Science Foundation, with Dr.

Senior Assistant Director for Graduate and A paper documenting this research, entitled “Material-Level Countermeasures for Securing

Post-doctoral Support, who played a key

Microfluidic Biochips,” was published as the cover story for the August 2023 issue of Lab on

role in bringing Baban and his colleagues

a Chip, a journal published by the Royal Society of Chemistry. According to Baban, the

to New York.

6


FA L L 2 0 2 3

THE RISING THREAT TO COMMUNICATIONS SYSTEMS While we tend to think of infrastructure

and secure NextG wireless systems

• Developing “a novel and powerful

in terms of large complex physical

from potentially unsecure hardware

evaluation platform to experiment

systems of steel and concrete, the term

components. As explained in the

with hardware security methods

also embraces less tangible structures,

NSF grant (see https://www.nsf.gov/

in both the baseband and RF in a

such as wireless communication

awardsearch/showAward?AWD_

high throughput millimeter wave

systems. Through partnerships with NYU

ID=2148293&HistoricalAwards=false),

software defined radio.”

Wireless and the Center for Advanced

the project focuses on “a particularly

Technology in Telecommunications,

important class of attacks called hardware

the Center for Cybersecurity is at the

Trojans, where hardware components

forefront of initiatives to protect emerging

supplied by a third party are maliciously

technologies in 5G and 6G.

altered to launch an attack from within

Currently at the heart of these efforts is a program funded by a $2.5 million grant from the National Science Foundation and involving three teams of researchers

a network node, such as a cellular base station.” The proposed solution to this threat includes: • Developing “computationally efficient methods to detect the

faculty Dr. Ramesh Karri and Dr. Siddharth

presence of hardware Trojans in

Garg. According to an NYU Research

both the baseband and RF”

was announced (see https://engineering. nyu.edu/news/three-nyu-tandon-teamswin-25-million-nsf-partnership-ensureresiliency-part-next-g-wireless), the goal of the initiative is to make “current and future wireless infrastructure, software and hardware systems more resilient to flaws, accidents, subterfuge and hacks.” One of the projects covered under

initiative at Tandon is that the defenses generated through the research were to be evaluated by participants in the 2022 CSAW Embedded Security Challenge. Funding for the program comes from NSF’s Resilient and Intelligent

at NYU Tandon, including CCS affiliated

Brief released in May 2022 when the grant

One interesting new wrinkle to the RINGS

• Estimating “the capacity of

Next Generation Systems (RINGS) program (https://new.nsf.gov/funding/ opportunities/resilient-intelligent-nextgsystems-rings), which is dedicated to initiatives that “accelerate research in areas that will potentially have significant

undetected hardware attacks” that can

impact on emerging Next Generation

enable “a critical optimization of the

wireless and mobile communication,

power and computation on hardware

networking, sensing, and computing

verification and potential throughput

systems, along with global-scale services,

degradation”

with a focus on greatly improving the

• Extending these methods to network settings, including jamming

resiliency of such networked systems among other performance metrics.”

and multi-user attacks.

the grant focuses on building resilient

Top left to right: Elza Erkip, Siddharth Garg, Zhong-Ping Jiang, Pei Liu, Farshad Khorrami. Bottom left to right: Sundeep Rangan, Ramesh Karri, Shiv Panwar, Yong Liu

7


FA L L 2 0 2 3

FACULTY PROFILE:

MICHAIL MANIATAKOS RE-THINKING CYBER DEFENSES FOR MODERN INDUSTRIAL CONTROL SYSTEMS Dr. Michail Maniatakos is an associate professor of computer engineering at NYU Abu Dhabi, where he also serves as director of the Modern Microprocessors Architectures (MoMA) Laboratory. His work at MoMA focuses on strategies to preserve security and privacy in high-performance and embedded microprocessors. To date, these initiatives have led to the development of novel privacy-preserving architectures that process data while encrypted, and security assessment solutions that utilize homomorphic encryption for applications in industrial control systems. He has received research grants from a number of industrial and governmental agencies, including the US Office of Naval Research, DARPA, and the Abu Dhabi Department of Education and Knowledge. Maniatakos completed his bachelor’s degree and a master’s degree in computer systems technology at the University of Piraeus in Greece, then went on to complete Master’s and Ph.D. degrees in electrical engineering at Yale University. A senior member of the IEEE, he has authored several publications for IEEE/ACM Transactions and conferences, and has served on the technical program committee for various IEEE/ACM conferences. Maniatakos also holds several patents for privacy-preserving data processing strategies. In this profile, he shares some insights into his work in protecting critical infrastructures.

8


FA L L 2 0 2 3

CyberByte: Your research seems to sit right at the intersection of

We included a series of recommendations in the paper by which ICS

software and hardware security. What led you to your current niche

vendors may be able to mitigate supply chain risk. These include

of research interests?

having vendors release extensive SBoMs, and mandating independent security verification of the third-party components they include in

Maniatakos: My main focus is cybersecurity research, that is

their product supply chains. We also call for expanded regulations to

protecting systems, networks, and people against cyber attacks and

make sure vendors continuously monitor vulnerability disclosures and

ensuring privacy is preserved. An attacker can use different points,

respond to high-ranking CVEs within a predefined number of days.

either at the software level or at the hardware level. But it’s rare that a single attack can be efficiently created targeting only an isolated

CyberByte: You have applied A.I. in a number of your research

abstraction layer. Hardware techniques need software support and

projects. How has access to these tools changed the approach to

vice versa. This is the main reason my research lies in the intersection

creating more secure industrial systems?

of software and hardware security.

Maniatakos: A.I. is a great tool to solve a specific subset of research problems, particularly when there is an enormous amount of noisy

CyberByte: Since this issue of CyberByte is focused on securing

data entering the system that needs to be processed fast. Application

critical infrastructure, can you summarize some of your specific research efforts in this area?

of AI to industrial control systems is not straightforward, since the

Maniatakos: We are trying to address the cybersecurity implications

a number of wrong outputs (false positives/negatives), which can be

of two industry trends. The first is the transition to “smart” devices,

severely problematic in industrial control system settings. Therefore,

or Industrial Internet-of-Things devices, which have added “hackable”

in this arena, AI is mainly used for data extraction/analysis from

devices into our critical infrastructure. Since the word “smart” implies

industrial control systems. Actual real-time solutions for detecting

the presence of regular microprocessors and operating systems,

and blocking cyberthreats still rely on precise rules and algorithms.

data produced is limited in quantity. Furthermore, AI typically exhibits

existing cyberattacks can easily be ported to industrial systems. The

CyberByte: Another recent paper “Perception, Performance, and

second trend is hyperconnectivity, which implies that everything

Detectability of Conversational Artificial Intelligence Across 32

connects to everything. This makes it much easier to get access to

University Courses” took a look at the impact of one A.I. tool,

sensitive devices and attack them.

ChatGPT. Can you summarize your findings from that study?

Because of these two trends we need to think about new defenses

Maniatakos: This is one of the first studies (probably the first) to

because existing strategies cannot be easily ported to such systems.

investigate the impact of AI generated text on universities. The study

They are limited by strict real time requirements, availability

collected existing student answers to questions from a variety of

requirements, and levels of computational power. Thus, my research

courses (32), and generated new answers from ChatGPT for every

efforts can be summarized as revisiting and creating new cyber

question. Then the mix of student and AI answers were given to

defenses for modern industrial control systems.

graders for evaluation. We concluded that ChatGPT’s performance is comparable, if not superior, to that of students in a multitude of

Another increasing threat to industrial control systems (ICS) is the

courses. Moreover, current AI-text classifiers cannot reliably detect

outsourcing of the software for the programmable logic controllers

ChatGPT’s use in school work, due to both their propensity to classify

(PLC) that make the systems run. Though there are a number of

human-written answers as AI-generated, as well as the relative ease

valid reasons for such a move, we have found that active third-party

with which AI-generated text can be edited to evade detection.

projects are updated regularly with public Common Vulnerabilities and Exposures (CVE) system disclosures and patch changelogs. In

CyberByte: Reviewing some of your other recent publications,

doing so, these projects provide a readily available attack vector

can you elaborate on a paper you published earlier this year about

to an adversary.

using homomorphic encryption to preserve privacy in cancer prediction strategies?

We recently published a paper “Dissecting the Industrial Control Systems Software Supply Chain,” in the July/August 2023 issue

Maniatakos: Predisposition to cancer can be identified by analyzing

of IEEE Security and Privacy, that assessed the current security

our DNA. If we know that we may be more susceptible to some form of

status of these systems, Along with recent Tandon Ph.D. graduate

cancer, we can perform the necessary lifestyle changes or, since cancer

Prashant H.N. Rajput and coauthors from the Georgia Institute of

can be more easily treated in its early stages, to monitor ourselves

Technology, we reverse engineered several ICS devices and indexed

regularly. But, our predisposition to some form of cancer can be used

their third-party components. Based on the study, we can draw a

against us. For example, health insurance providers could skyrocket

few conclusions. First and foremost, the risk of using third-party

our premiums to force us to drop their insurance, so they won’t have

code is real. Second, many vendors are incorporating open source

to pay for our treatment in the future. Therefore, we want to be able to

components into their firmware, a choice which brings with it a

get answers about predicting cancer while maintaining privacy for the

community of users willing to contribute back to the projects. And,

patients. We are able to do that using a new form of encryption called

third, it confirms the need for vendors to invest time and resources in

fully homomorphic encryption. This special type of encryption allows

maintaining legacy code, particularly as we also found that updates

us to compute data without revealing it to anyone else, which is the

will continue to be slow.

strategy we use to preserve privacy in cancer prediction.

9


FA L L 2 0 2 3

serves undergraduate students in his

system will use only a small fraction of

home country of Nepal (https://cyber.nyu.

that code. Currently, the approach of many

edu/2020/09/01/tandon-ph-d-student-

applications is to either turn on security,

launches-virtual-cybersecurity-mentoring-

which slows things down a bit, or turn

program-for-nepali-undergrads/). Named

off security all together. Users look at the

after the Nepali word for the horizontal

trade-off and see if they’re willing to take

wooden bar that secures doors in

the performance hit. Since we are working

traditional houses, Prasant has overseen

on only a small part of the code, we believe

Gajabaar for the past four years. This past

that what runs is generally bug free. So, the

spring, Prasant sat down for an interview

TRACKS project asks the question “Can

about his current research initiatives

we just enable security in the part that

and about the continued success of the

does not run, but not bother with the part

Gajabaar program.

that runs all the time?” The project aims

CyberByte: You switched advisors and your research focus after being at NYU Tandon for several years. Did the move

PH.D. PROFILE:

PRASANT ADHIKARI PROTECTING THE LINUX KERNEL, STRENGTHENING CYBERSECURITY IN NEPAL Prasant Adhikari came to NYU Tandon for his doctoral studies following completion of a bachelor’s degree at NYU Abu Dhabi in 2018. He began his time at Tandon working with Dr. Ramesh Karri on hardware Issues. Earlier this year, he shifted his research interest to software and network security and began working under the supervision of Dr. Justin Cappos

to get the best of both worlds, to get the performance where we need it, and the security where we need it.

from a hardware to a software research

CyberByte: The TRACKS project is being

focus require a change in mindset?

run under an NSF grant, so what is your

Adhikari: In security, we talk a lot about threat models and how the approach you

time limit? When are you expected to complete the work?

take changes with your threat model. In

Adhikari: We expect to have deliverables

hardware, part of that threat model is

in the next two years. The end goal is to

what manufacturers could have carried

produce something that can be adopted in

into the hardware. Whereas, if you are

the industry right away.

working with operating systems, you are not going to worry too much about that problem. The threat model changes and you start thinking about your system going out to other people, and if they could do something malicious. CyberByte: You are currently working on the TRACKS (“TRimming Augments Container Kernel Security”) project, an NSF grant project that supports improved container security by identifying which portions of the Linux kernel are most likely to be vulnerable. Can you briefly describe

CyberByte: I understand you are currently serving an internship. Can you tell me a bit more about it? Adhikari: I’m working for an Austin, TXbased security consulting service called Praetorian. Not every company has a security team, so this company audits security and sees where the clients are vulnerable. And then, for clients who don’t have security, they do the monitoring throughout the year. At this time, I’m writing some modules that would automate

the project and your work on it?

testing for vulnerable clients. Though there

has previously been profiled on the CCS

Adhikari: The idea is actually kind of

my current research, there is some benefit

website for founding a summer training/

simple. The Linux kernel is a massive core

in threat hunting. Part of what my research

mentorship program called Gajabaar that

base with millions of lines of code, but any

requires is looking at exploits that people

on improving container security. Prasant

10

is no direct tie between the internship and


FA L L 2 0 2 3

have written and asking if our solution

program. We now receive about 35 to 40

there have been a few attacks on banks

would have prevented those things.

applications each semester The program

that resulted in new regulations. Something

is also now less managed and more

that Nepal is doing quite well with is bug

independent. We’re doing less one-on-one

bounty hunting, where individuals identify

work than when we began.

and report bugs in code.

iteration of the program. What progress

CyberByte: In the four years you have

CyberByte: One last question: What

have you seen in the program since then?

been running the program, have you seen

happens with Gajabaar once your academic

a greater awareness of cybersecurity in

career is over?

CyberByte: Let’s talk a little bit about Gajabaar. When we last talked in the Fall of 2020, you had just finished your first

Adhikari: We’ve seen some growth in size. In 2022, I was able to do a presentation

Nepal?

Adhikari: I’m trying to arrange things so

about the program at ThreatCon ‘22, an

Adhikari: I would say that, in general,

that those who go through the program

annual cybersecurity event held in Nepal

awareness in Nepal has gotten better. As

will be encouraged to give back.

since 2018 (see https://threatcon.io/

I mentioned, the country is now doing a

about) which drew some attention to the

cybersecurity conference each year, and

11


ALUMNI PROFILE:

ZAHRA GHODSI CRAFTING A HOLISTIC APPROACH TO TRUSTWORTHY SYSTEMS Zahra Ghodsi came to NYU Tandon from Sharif University in Iran, where she did her undergraduate work. During the years between 2015 and 2020, she completed Master’s and Ph.D. degrees in electrical and computer engineering at Tandon. After a postdoctoral appointment at the University of California-San Diego, where he worked with Prof. Farinaz Koushanfar, she accepted a post as an assistant professor of electrical and computer engineering at Purdue University. Zahra is a recipient of the AI Research Fellowship from J.P. Morgan, and the Ernst Weber Fellowship from the Tandon Department of Electrical and Computer Engineering. CyberByte: Can you describe in the simplest terms possible your particular research focus? Ghodsi: My research broadly focuses on building trustworthy systems, most recently developing solutions for private and secure collaborative learning. Accessing, processing, and learning from the massive amounts of data available today can enable a multitude of new useful applications, but requires addressing issues like privacy of sensitive information and correct behavior in the presence of errors or even attacks from malicious actors. I particularly enjoy designing systems with rigorous security and privacy guarantees that are based on cryptographic primitives and protocols. This can make existing systems more secure or enable new applications. For a complex ecosystem, such as collaborative learning with mutually distrusting parties, it is very challenging to build secure

12


FA L L 2 0 2 3

systems that are both practical and scalable. I believe that advances

CyberByte: What have you enjoyed about teaching? And, what

in cryptography, hardware systems, and learning paradigms,

challenges have you had to overcome? Has anything in particular

and their combination and co-design together, can allow us to

surprised you in the classroom?

achieve the requirements of real-world systems. The intersections

Ghodsi: In Spring 2023 I taught an undergraduate class on

and interactions of these fields are at the heart of my research

Microprocessor Systems and Interfacing. Teaching a relatively large

aspirations.

class (200+ students) in my first semester as an Assistant Professor was intimidating at first, but I quickly eased into my teaching role;

CyberByte: How did you initially become interested in this idea of

I have to thank our amazing staff and TAs for that! I really enjoyed

building secure frameworks for intelligent systems? And was there

my interactions with students inside and outside of the class. Purdue

any particular project or event (such as an internship) that may have influenced the direction of this research during your time at NYU?

has an outstanding undergraduate engineering program, and it was

Ghodsi: A few classes that I took during my master’s and early PhD

happy to see that students reflect the energy and enthusiasm of the

(e.g. by Prof. Ramesh Karri and Prof. Patrick Cousot) spiked my

instructor, and that providing a bit of context and history about the

interest in computer security. As I learned more about cryptography

topics covered goes a long way in attracting students and keeping

and its beautiful constructions, I started to gravitate towards

them engaged.

a pleasure to spend a semester with so many bright students. I was

problems that occur when applying cryptographic protocols in

CyberByte: After spending so many years in urban environments

interesting emerging applications, such as collaborative learning. I

(first NYC and then San Diego) I’m wondering how you are liking the

have to credit my wonderful advisor, Prof. Siddharth Garg, for giving

quieter pace of life in the midwest. And what, if anything, do you

me the space and guidance to explore directions and problems that

miss from life in a big city?

I found most interesting during my Ph.D.

Ghodsi: There are definitely differences between life in NYC

CyberByte: After graduation, you did a postdoctoral assignment at

and West Lafayette, but I was pleasantly surprised to find many

the University of California, San Diego. What attracted you to this

aspects of life that I enjoyed in NYC do exist here as well. I loved

position and what do you think it contributed to the direction of

the diversity of people in NY, and I am happy to have found social

your work now?

groups and connections with people from all walks of life in Indiana. I miss some of my regular spots, like the IFC Center for independent

Ghodsi: I had a great time as a postdoc scholar at UCSD, and got

films, and I miss being able to grab a bite from a favorite restaurant

a chance to expand my research, interact with industry partners,

at an ungodly hour. However, I am pleased to find more local

and work with many wonderful students in an advisory role. I was

businesses here, where owners make personal connections with

attracted to the exemplary track record of the group led by Prof.

customers easily and seem to enjoy a better balance between work

Farinaz Koushanfar, its broad span of ongoing research activities,

and life.

and its close collaborations with other research groups. Connecting with industry people and discussing problems they face has further

CyberByte: I was very pleased to see you are still frequently

guided my research towards real-world systems and targeting of the

collaborating with former NYU colleagues, including most recently

pressing challenges facing them today.

a collaboration with Ghada Almashaqbeh about protecting user anonymity in federated learning. How did this research collaboration

CyberByte: Your past work has included forays into developing test

come about?

scenarios for autonomous vehicles and several initiatives for dealing with performance issues from privacy strategies. How did these

Ghodsi: Indeed, I continue to be in touch with former colleagues

particular research initiatives evolve? And, how do they fit with your

and contacts from NYU and Columbia, and on some occasions,

overall research focus?

get to collaborate with them! Ghada leads a research group at the University of Connecticut and is already making a mark as a junior

Ghodsi: As I continued to explore urgent concerns in real-world

assistant professor with an outstanding profile. Ghada and I have

complex learning systems, I started to think about robustness

the perfect balance of complementary and overlapping expertise

and the safety requirements that augment security. When we

and had been informally discussing some ideas for a while. After

talk about a “trustworthy system,” we should think about many

we both got a chance to independently lead projects, we were

aspects, such as security, privacy, robustness, and safety, in addition

excited to work on some of those ideas! This is one of the aspects of

to mechanisms for verification, testing, and auditing. During an

academic life that I enjoy the most, the flexibility to work on a range

internship at NVIDIA, I worked on developing testing methods

of real-world problems with brilliant students and collaborators.

for autonomous vehicles. In that industry, safety concerns are a primary issue. However, building a trustworthy system necessitates meeting many requirements at once, which can sometimes look contradictory. For example, how do we guarantee security and privacy, but also enable testing and auditing? In my future research, I plan to address building trustworthy systems in a holistic manner.

13


FA L L 2 0 2 3

(Left to Right): Phil Venables, Chief Information Security Officer of Google Cloud; Shih-Fu Chang, Dean of Columbia Engineering; Greg Morrisett, Dean and Vice Provost of Cornell Tech, Cornell University; Kurt Becker, Vice Dean of Research, Innovation and Entrepreneurship, NYU; Joshua Brumberg, Dean for the Sciences at The CUNY Graduate Center. Photo courtesy of Google.

CCS NEWS GOOGLE COMMITS $12M TO ENSURE THE BIG APPLE’S PRIMACY IN CYBERSECURITY: On June 12, Google rolled out the Cyber NYC Institutional Research Program, an ambitious initiative to “bolster NYC’s cybersecurity leadership through University programs.” As described in a press release issued by NYU, the program will provide annual funding of $1 million for the next four years to a quartet of academic institutions in the NYC Metropolitan area, including NYU Tandon. The goal of the grant is to support “cutting-edge research” and to “expand educational opportunities for students seeking advanced degrees in cybersecurity.” As part of this commitment, Google will fund research through 2025 at The City University of New York, Columbia University, Cornell University (including Cornell Tech and the Cornell Ann S. Bowers College of Computing and Information Science), as well as New York University. According to the NYU release, the funding will support more than 90 collaborative research projects in areas where “further research could encourage the development of more secure digital ecosystems and inspire innovation.” In addition, the schools can use these funds to “expand and improve their cybersecurity degree programs,” which in turn can “fortify the incoming cyber workforce, and also address the diversity gaps in the industry by focusing on recruiting and developing workers from underrepresented groups.” Speaking for NYU Tandon School of Engineering, Dean Jelena Kovačević observed, “Unsecure cyber space is one of the greatest threats to our society and the only way to be sure to create safer information systems, networks and communications is by bringing academia and industry together. This funding from Google will allow our Center for Cybersecurity, as well as partners at other NYU schools and across the city, to tackle these important threats and focus on a safer future for everyone.” You can read more about the program at: https://engineering.nyu.edu/news/googleannounces-12m-research-program-local-universities-bolster-nycs-cybersecurity-leadership

14


FA L L 2 0 2 3

CSAW CSAW MARKS ITS 20TH ANNIVERSARY THIS NOVEMBER In 2003, Professor Nasir Memon started a small local Capture the Flag competition for students involved in NYU Tandon’s cybersecurity program. Twenty years later, the event became Cybersecurity Awareness Week, and later just CSAW. It has expanded to 5 global academic centers, and its cyber competitions, in the words of its director, Dr. Ramesh Karri, continue to

MEMON ASSUMES NEW RESPONSIBILITIES AT NYU SHANGHAI

“evolve to keep pace with

Dr. Nasir Memon, NYU Tandon Vice Dean for

based AI.”

Academic and Student Affairs, Professor of Computer

advancing technologies, such as additive manufacturing, machine learning, and cloud-

Science, and department head of NYU Tandon Online,

This year’s edition of CSAW, to

has begun a new challenge this fall. On June 9, 2023,

be held from November 8-11,

NYU announced that Memon would be assuming the post of Interim Dean of Computer Science, Data Science, and Engineering at NYU Shanghai. The new assignment offers Memon a chance to further build on his impressive record of achievements as a faculty member and administrator at NYU. In addition to the titles mentioned above, Memon is also a co-founder of NYU’s Center for Cybersecurity, and a driving force behind CSAW, the world’s most comprehensive student-led cybersecurity event.

will acknowledge the growth and accomplishments of what is now recognized as the largest student-run event of its kind. While at the time this written most activities were still in the planning stages, most of the competitions,

According to the announcement issued by NYU, in his new role Memon will be responsible

including Capture the Flag,

for the development of academic programs in computer science, data science, and

Embedded Security, Hack

engineering at both the undergraduate and graduate levels. In addition, he will be helping these programs “develop their curriculum, build their faculty, and operate as true communities of scholars.”

3D, and Logic Locking, have already opened registration or are set to hold preliminary

In accepting the post, Memon noted that NYU Shanghai offers him an “opportunity to

competitions shortly. The

nurture the next generation of innovative minds, and shape the academic landscape

CSAW website at https://www.

in an international setting.” He adds that, “the unique combination of NYU’s renowned reputation, the dynamism of Shanghai as a global tech hub, and the chance to work with talented faculty and students from diverse backgrounds create a compelling environment for personal and professional growth.”

15

csaw.io/ is the best place to stay abreast of emerging plans for the event.


FA L L 2 0 2 3

RECENT CCS MASTER’S GRADUATE SERVES AS LEAD FOR CS4CS SUMMER PROGRAM For several summers, NYU Tandon has sponsored the Computer

helped revamp the cybersecurity curriculum by integrating and

Science for Cyber Security (CS4CS) program, a three-week workshop

conducting custom Capture the Flag competitions. He reports that

run though NYU’s K12 STEM center. The program is open to students

the program served about 80 students in total, selected from a pool

in grade 8-12 in New York City and surrounding areas. Over the years

of more than 700 applicants.

CS4CS has explored such topics as white-hat hacking, cryptography, steganography, digital forensics, privacy, and data usage.

In the third week, representatives from program sponsors DTCC

This year, the program was led by Staford Titus, a recent M.S. graduate

real world cybersecurity landscape and to provide mentorship and

of the NYU Center for Cybersecurity. In addition to training and

internship opportunities to the students.”

and Netskope visited to, in Titus’s words, “share more about the

managing six instructors that worked directly with the students, Titus

16


FA L L 2 0 2 3

EVENTS NY GOVERNOR HOCHUL CHOOSES NYU TANDON TO INTRODUCE FIRST STATEWIDE CYBERSECURITY POLICY On August 8, Governor Kathy Hochul introduced New York State’s first plan to protect

The Tandon campus was chosen for

digital infrastructure in a special presentation at NYU Tandon School of Engineering in

the announcement in recognition

Brooklyn. According to an article by Jada Camille that appeared in Brooklyn Paper.com

of the school’s long commitment

(https://www.brooklynpaper.com/hochul-statewide-cybersecurity-strategy/), the plan

to cybersecurity, and the training

“outlined how the state will protect critical data, networks, and technology systems

partnerships that have been formed

against cyber threats — securing both New Yorkers’ personal information and government

between the school and the state, Hochul

data and operations.”

observed that there is a need for more

“ Our interconnected world demands an interconnected defense leveraging every resource

and computers. We don’t just need A.I.,

available,” Hochul said in a statement describing the policy. “This strategy sets forth a

we need H.I–Human Intelligence.” To

nation-leading blueprint to ensure New York State stands ready and resilient in the face

illustrate this point, a group of graduates

of cyber threats.”

from the NYU Tandon Digital Learning

The plan, as described in a factsheet from the Governor’s office (see https://www.governor.

Operational Technology Security were

ny.gov/sites/default/files/2023-08/2023-NewYorkCybersecurity_FactSheet.pdf) is built to

invited to the event. Over the last few

ensure the state’s cyber systems and its operators are:

months, at least 60 employees of the

than “just systems, digital platforms,

MTA Professional Learning Certificate in

Metropolitan Transit Authority (MTA) have •U nified, by working to increase access to cybersecurity information, tools, and

received cybersecurity training.

services so that the State’s most sophisticated defenses are available to its least wellAlso speaking at the event were Janno

resourced entities.

Lieber, CEO of the MTA; Jake Braun,

•R esilient, by moving to expand the scope of cybersecurity regulations, requirements,

Acting Principal Deputy National Cyber

and recommendations so that New York’s critical infrastructure is better protected.

Director for the Biden Administration; and philanthropist Craig Newmark.

•P repared, by providing advice and guidance to ensure New Yorkers are empowered

The entire presentation is available for

to take charge of their own cybersecurity unification, resilience, and overall

viewing at https://www.youtube.com/

preparedness.

watch?v=XmjTXgx3X4s.

17


FA L L 2 0 2 3

IEEE HOLDS MAY DIGITAL MANUFACTURING WORKSHOP AT TANDON On May 1 and 2, NYU Tandon welcomed to campus the Second Annual IEEE Workshop on Reliable and Resilient Digital Manufacturing. The two-day event included lectures and panels on a wide range of topics relevant to this growing manufacturing sector, including security, artificial intelligence, technological advances, and future implementations. Organized by Professors Nikhil Gupta and Ramesh Karri of the NYU Center for Cybersecurity, in partnership with Professor Nektarios Tsoutsos of the University of Delaware’s Center for Cybersecurity, Assurance and Privacy, the workshop featured

CYBER FELLOWS ADVISORY COUNCIL HOLD ANNUAL MEETING

keynote talks by Dr. Andrew Wells, Program Director in Advanced Manufacturing at the National Science Foundation; Dr. Yan Lu, Information Modeling and Testing Group Leader at the National Institute of Standards and Testing; Dr. Ronald Paveda, a former doctoral student at NYU Tandon and now a researcher at the Naval Air Systems Command; and Dr. A. Narasimha Reddy, a professor of electrical and computer engineering at Texas A&M University.

In April, 2023, members of the NYU Tandon Cyber Fellows Advisory

Specific session topics included balancing 3D printing opportunities and security in

Council held its annual meeting.

the case of aftermarket parts, designing anti-counterfeiting schemes for mechanical

The 50-member council is a group

parts, and proposing ways in which additive manufacturing can be leveraged by the

of CISOs and industry leaders

solar-energy industry.

who engage with the Tandon community in multiple ways. It includes corporate partners and industry stakeholders who enhance the ability of the program’s industry collaborators to influence and shape the cyber community. The meeting brought together an esteemed group of cybersecurity leaders and guests for an interactive session aimed at tackling the ongoing critical workforce needs of the industry. Topics addressed ranged from upskilling and reskilling employees, to diversity, equity and inclusion initiatives. The program culminated in a networking session for Master’s students that featured executives from Bank of America, TIAA, BNY Mellon, and Collins Aerospace. “ We take great pride in our robust and deep industry relationships, and we are grateful to our Council members who support NYU Tandon and our Digital Learning efforts” says Shivani Dhir, Assistant Dean of Digital Learning. “We remain committed to ongoing collaboration to ensure we deliver academic programs and pathways that help shape the future of cybersecurity.”

18


FA L L 2 0 2 3

(pictured from l to r): Kylie Watson, Sumitomo Mitsui Banking Corporation; Jason Harrell, Depository Trust & Clearing Corporation (DTCC); Edward Amoroso, TAG Cyber; and Joel Caminer, NYU Center for Cybersecurity

FIGHTING BACK AGAINST THANOS AND MORE: THE 13TH ANNUAL CYBERSECURITY LECTURE

Using these familiar characters as avatars

didn’t lose until all of Marvel’s heroes

for threat actors, Harrell made it clear

fought him together.

Anyone who signed up to hear keynote

Harell points to two statistics that

speaker Jason Harrell from The Depository Trust & Clearing Corporation (DTCC) at the 13th AIG-Sponsored Cyber Security Lecture might have expected a dry description of the current threat landscape and prescriptions for navigating it. Instead, attendees at the June 1 lecture, which was co-sponsored by the NYU Center for Cybersecurity, were treated to a colorful and effective talk that used a collection of Marvel comic villains to describe today’s cyber threat actors. At the top of that heap of baddies was Thanos, who Harrell likened to well-funded, persistent, and highly skilled nation-state threat actors like those involved in the 2007 Estonia Cyber Attacks, the 2010 Stuxnet Attack, and the 2016 Bank of Bangladesh Heist, to name just a handful.

that the chance of an attack affecting numerous systems—including vital financial-sector entities—is now inevitable. confirm that result. First, he observes that more money is now made each year in cybercrime than in narcotics, and second, that it takes an average of just 84 minutes for an adversary to move laterally from an initial compromise. Therefore, Harrell emphasizes that it has become imperative that the cyber ecosystem be as resilient as possible, with resilience being defined as “the ability to protect, detect, respond to and recover from operational incidents,

The event concluded with a panel discussion moderated by Joel Caminer, senior director at the NYU Center for Cybersecurity, that also featured Kylie Watson, the CISO of Sumitomo Mitsui Banking Corporation-International Bank, and Ed Amoroso, the CEO of TAG Cyber. Among the points made by the group was that the cyber landscape is evolving so rapidly, that one does not need decades of experience to make a mark in the field. Since cyber professionals at every level must continually expand and update their knowledge base, it opens the field

including cyber attacks.”

for a wider number of professionals with

Harrell emphasized that creating resiliency

and in terms of work experience

will involve leveraging a more diverse talent pool, and making effective use of emerging technologies like AI. Creating strong public-private partnerships is also key since, to continue the analogy, Thanos

19

diverse backgrounds, both academically

To watch the event in its entirety, which also included remarks from AIG’s Ed Hayes, go to https://www.youtube.com/ watch?v=XiRcyKnlcj8.


FA L L 2 0 2 3

AWARDS AND HONORS

seen on some microcontrollers developed

KARRI NAMED AN INTEL OUTSTANDING RESEARCHER

team at NYU Tandon that includes Ph.D.

using the ‘Chip-Chat’ methodology.” This strategy, developed by a research student Jason Blocklove, and Drs. Ramesh Karri, Siddharth Garg, and Pearce, is documented in a 2023 paper published in Machine Learning (see Arvix version at https://arxiv.org/pdf/2305.13243.pdf). To read more about Pearce’s winning design, go to https://efabless.com/hammond-firstplace-winner.

Ramesh Karri, co-founder and co-chair of NYU’s Center for Cybersecurity, was honored in the spring of 2023 by Intel as one of the previous year’s seven outstanding academic researchers. Recognizing “exceptional contributions made through Intel university-sponsored research,” Karri was selected for his work on a project called “The Path Towards System-on-Chip Survivability.” In a press release from NYU announcing the award, Karri explains the project as follows: “In the world of software, if a vulnerability is discovered, it’s easy to provide a patch. It’s different with hardware; you must detect any vulnerability before the chip is actually fabricated.” His answer to this challenge is to build “Patching Blocks,” which can “leverage field-programmable gate arrays” to “monitor security bugs and perform corrective actions.” In congratulating Karri on this accomplishment, NYU Dean Jelena Kovačević“ noted, “This latest honor highlights the importance of his research, which has cemented NYU Tandon as a world leader in the vital realm of hardware security.” To read more about the award, go to https://engineering.nyu.edu/news/ nyu-tandon-professor-named-one-intelsoutstanding-researchers-year

NYU COMPETITORS EXCEL IN 2023 HACK @ DAC AND NEW EFABLESS DESIGN CHALLENGE A cross-continent team of competitors from NYU Tandon and the University of New South Wales in Sydney, Australia, took 4th place in the 2023 “Hack @ DAC” competition, held in San Francisco from July 9 to 13. The team, called the “NYU_ bounty_hunters” was led by Dr. Hammond Pearce, who has just assumed a post as a Lecturer at the University of New South Wales after serving as a visiting scientist at NYU Tandon for three years. Team members include Prithwish Basu Roy, Meet Udeshi, Animesh Basak Chowdhury, and Jason Blocklove, all from NYU. See the full list of winners at https://www.linkedin. com/pulse/hackdac-2023-winners-jason-

ABOUELNOUR CAPTURES FELLOWSHIP AWARD Youssef AbouelNour, a Ph.D. student in mechanical and aerospace engineering who works with Dr. Nikhil Gupta on cybersecurity issues, won a 2022 American Society for Nondestructive Testing Fellowship Award for his research on the topic “In-Process Non-destructive Testing and Evaluation for Defect Detection in Additive Manufacturing.” The fellowship includes a $20,000 cash award, publication of a paper on the completed

fung/.

research in one of the ASNT journals, and Dr. Pearce recently garnered another

a presentation at an annual conference.

honor when he took first place in the Efabless AI Generated Open-Source

During his tenure at Tandon to date,

Silicon Design Challenge. According

AbouelNour has also published a research

to the official Efabless announcement,

article, “Assisted Defect Detection

the competition asked participants to

by In-process Monitoring of Additive

“design and tapeout an AI generated

Manufacturing Using Optical Imaging

open-source silicon design.” His winning

and Infrared Thermography” in Additive

design, called QTCore-C1, is “a co-

Manufacturing, and a review article in

processor that can be used for many

Materials & Design.

applications, such as predictable-time I/O state machines for PIO functions as

20


FA L L 2 0 2 3

AWARDS AND HONORS ROUND-UP: CONGRATULATIONS TO: • Dr. Brendan Dolan-Gavitt, who was elevated to the title of Associate Professor at NYU in March. • 2022 ECE Ph.D. grad Linan Huang who was honored with one of the 2023 Dissertation Awards in Science and Technology. The award honors his research work on “AI-Powered System-Scientific Defense for HighConfidence Cyber-Physical Systems: Modeling, Analysis, and Design,” which focused on the development of high-confidence cyberphysical systems to protect vital information and national security. Writing in the abstract, Huang states, “Drawing on philosophies, theories and tools from multiple disciplines, as well as CPS datasets and human research studies, I laid the scientific foundation for the fifth-generation security paradigm,synthesizing six revolutionary transitions.” • Yunian Pan, a Ph.D. candidate who works with Dr. Quanyan Zhu, was one of 5 recipients of the 2023 Dante Youla Award for Graduate Research Excellence in Electrical and Computer Engineering.This award is given to the graduate student who has “made the most significant research contribution among all ECE graduate students.”

21


FA L L 2 0 2 3

CENTER FOR

CYBERSECURITY cyber.nyu.edu

22


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.