SMART GLASS & ARTIFICIAL INTELLIGENCE Manoj Phatak - Founder of ArtRatio and Smart Glass World

Smart Glass Artificial Intelligence

Smart Glass Artificial Intelligence Artificial Intelligence

Smart Glass is only as smart as the control system that drives it. Whether it is for viewing the exterior world from within buildings or vehicles, or filtering incoming daylight onto interiors, smart glass systems will need to evolve to a state where they are continuously learning from their immediate environment in order to do their job better.

Manoj Phatak Founder of ArtRatio and Smart Glass World

Let me explain

Your smartphone can adjust its behaviour based on your personal data and local events, right?

Examples include: • dimming down the screen when the battery is low, • detecting the risk of a heart attack using a medical app, and • finding the best sushi restaurant, near you, right now. happening in its local environment and what it is learning from global events as they unfold in real-time.

Use Cases

We have seen some excellent examples in the news recently from smart glass manufacturers such as View and Halio in which proprietary artificial intelligence (AI) algorithms adapt building facades to changes in sunlight, with clear benefits (pun intended) for sustainability, privacy and comfort.

Similarly, a smart glass system should also be able to adjust its behaviour based on what is Let us now consider some more advanced use cases that would take this one step further:-

Security Threats in Large Cities

Just as car doors unlock automatically on detecting an accident, smart glass facades could also default to a transparent state in emergency situations.

For example, if an AI agent monitoring social media chatter detected an imminent terrorist threat in a large city during a bright sunny day, it is probable that any building with a smart glass facade would be in its energy-filtering ‘dark mode’, to minimise glare and air conditioning costs.

This dark mode however would also impair visibility into the building for law enforcement and security agencies trying to ensure everyone has been evacuated from the building.

In this case, smart glass facades in the immediate vicinity of the security threat could automatically transition to an ‘emergency mode’ in which they are held indefinitely in a transparent state, aiding visual communication.

The same principle could also be applied to smart glass systems installed on transportation, enabling a visual line of communication with passengers trapped after an accident, or when the vehicle predicts the risk of a stranded infant or pet suffering possible dehydration.

Theft of High-Value Items

Whether in a private residence, a high-end jewellery store or an art gallery, if smart glass is being used on internal walls or in showcases, and if the AI algorithm monitoring CCTV cameras anticipates an imminent theft, the smart glass system could ‘go dark’, obscuring the items and slowing down the attacker from finding the most valuable pieces.

When the premises are closed, security guards still need to be able to see the displayed items while on patrol. However, if the building security is likely to be breached, the sensible solution would be to hide everything from view. Not just the most valuable items, but everything.

It may not necessarily stop the attacker but it can slow them down, giving security or law enforcement agents extra time to respond.

Reducing Light Damage in Museums and Luxury Retail

Global art and luxury retail inventories worldwide can amount to Billions of Dollars in assets, but excessive light exposure can cause severe damage to paintings, fashion textiles and even fine wines, thus hurting their market value.

If the smart glass control system ‘knew’ which materials were on display (and their sensitivity), it could dim the glass down a few notches.

This change in illuminance would be practically imperceptible since the eye is an approximately logarithmic sensor (Weber–Fechner law), but it would substantially reduce the long term damage from light, since light exposure is the time-integral of the light level.

(This is what ArtRatio display vitrines do, by the way, but more on that later.)

Future smart glass systems should also learn to automatically identify which items are most vulnerable to damage using machine vision, then adjust their behaviour proactively and intelligently to preserve their value.

Let’s now take a step back and break down the basic building blocks to understanding such AI-driven smart glass systems:-

Firstly, What is Smart Glass?

Smart glass is not a product, but rather a family of technologies. Broadly speaking, this family can be divided into two major groups: ‘passive’ and ‘active’.

In both cases, a stimulus triggers changes in certain properties of smart glass, such as its transmittance, reflectance, refraction (i.e. scattering) or its electrical conductivity.

Passive smart glass reacts to an environmental stimulus such as light or temperature. Examples include thermochromic and photochromic smart glass which dim when struck by heat or light (respectively). Active smart glass, on the other hand, is driven electrically, by way of sensors or control commands from a building management system (BMS).

Examples of active smart glass technologies include: • suspended particle devices (SPD-SmartGlass), created by Research Frontiers Inc. • liquid crystal technology such as Merck eyrise or Smartglass International (note: when the crystals are dispersed in a polymer film this technology is called ‘PDLC’ smart glass),

• electrochromic glass (examples include View,

Halio, SageGlass and Heliotrope) and • micro-blinds, created by the National

Research Council of Canada (NRC).

You can find more on Smart Glass World, a site we created to educate our customers in smart glass technologies.

Within the active smart glass category, we could also reasonably include ‘transparent photovoltaic’ (TPV) glass, which converts part of the electromagnetic spectrum into electricity (albeit not as efficiently as solar cells). The advantage is that TPV glass is - well - transparent, thus allowing its use as windows in exterior building facades. When incorporated directly into the building facade, it is often referred to as Building-Integrated Photovoltaic (BIPV) glass.

Other materials which can be classified within the ‘smart glass family’ include smart mirrors, augmented-reality spectacles (think Google Glass or Apple Glass) and existing heads-up displays (HUDs) for transportation.

Where does Artificial Intelligence fit into all this? Artificial intelligence (AI) aims to predict future outcomes based on historical data, rather than relying on pre-programmed rules.

The system is ‘trained’ to recognise patterns, behaviours or properties, and over time learns what response to execute.

Artificial intelligence (AI) is often seen as an umbrella term, encompassing Machine Learning, which can be specialised to Deep Learning and even further specialised to Neural Networks.

For the purposes of this article, we will not delve further into the differences between these sub fields.

Current examples of AI applications include automated stock trading, customer service chatbots, Netflix’s recommendation engine and self-driving cars.

The key aspect of AI systems is that they adapt in real time, as opposed to traditional systems which are pre-programmed to obey a static set of rules.

AI algorithms can drive smart glass facades or products more intelligently, and even preempt certain dangerous or undesirable circumstances.

Can Smart Glass learn?

Smart glass itself is just a material, like concrete, stone or wood, with the key difference being the change in its optical properties when driven by a stimulus, as described above.

However, when coupled with an AI system, the whole ‘smart glass system’ can indeed learn, but this depends on whether there is a dataset to teach the system what is ‘right’ and ‘wrong’.

And this depends on whether there are sufficient real-time sensors recording the events which feed this dataset.

Based on the use cases outlined above, these ‘sensors’ might include:

• bots that monitor social media chatter, alerting law enforcement agencies to increased risks of a threat to the public; • a computer vision system recognising a known felon in the vicinity of likely targets of theft; • light exposure monitors that predict an increased risk of damage to light-sensitive materials such as pharmaceutical drugs, phototoxic oils, perfumes and dyes.

At ArtRatio, we are not yet using AI but rather a modified version of a PID (Proportional–Integral–Derivative) algorithm, similar to what you would find in a cruise control system in a car.

Our algorithm is adaptive. It is real-time. But it is not AI. that we were granted a European Patent for it in 2021.

If you subject an unpainted stone sculpture to light levels of say 500 Lux, it will not suffer noticeable deterioration.

But if you put a silk antiquity under those same light conditions, it can deteriorate in a matter of days or weeks. For this reason, ArtRatio smart glass vitrines modify their behaviour in real time based on what is being displayed, as well as local environmental variables and visitor engagement.

Some materials can be responsive to only temperature or humidity, but since about 50% of solar radiation is infrared (i.e. heat), incoming sunlight can increase the local temperature,

which decreases the relative humidity in an enclosed space.

So, light, temperature, humidity and even the electrical conductivity of air are all connected. But the damage is often triggered by light. And since active smart glass is an electricallycontrolled light filter, it seems obvious to use it to do just that.

ArtRatio Case Studies

ArtRatio has implemented smart glass display vitrines for museums, private collectors and luxury retailers, including most recently a private library in Boston and an exhibition of patrimonial jewellery for a major luxury house, which is due to take place in China in the summer of 2022.

Unfortunately, non-disclosure agreements prevent me from saying more right now, but stay tuned to our social media channels to see photos and videos when the abovementioned work can be officially revealed.

Our customers include the National Museum of Sweden, Sotheby’s Institute of Art and the Wellcome Trust.

The ArtRatio patented algorithm and our cloud-based analytics platform automatically balance the exhibition with the conservation of items, allowing our customers to decide whether to display, conserve, store, loan or sell the items.

What does the future hold for smart glass?

Our journey into adaptive, truly intelligent smart glass building facades, smart glass showcases and smart glass enabled transportation has just begun. We have merely scratched the surface of the available use cases.

Continued advances in materials and algorithms will benefit sectors where smart glass is already being used; from transportation to retail; from hospitality to heritage to healthcare.

What is exciting is that global initiatives such as sustainability and user privacy are driving us to a future that will be beautifully illuminated by smart glass.

Manoj Phatak

Manoj’s first experience with glass goes back to the fabrication of optical waveguides as a final year project at the Optoelectronics Research Centre (Southampton University) in 1989.

A sponsored student with Ferranti Semiconductors in the UK, Manoj worked thereafter as a Trainee Patent Agent in London, representing semiconductor firms at the UK, European and US patent offices.

Manoj founded ArtRatio in 2008 to build smart glass display cases for museums and luxury retailers, and launched SmartGlassWorld thereafter to promote development in smart materials.

Manoj’s experience as a manufacturer of smart glass end-products and as a smart glass distributor & consultant allows for a deep understanding of what customers need in order to achieve a return on investment in smart glass technologies.

Manoj is a UK Chartered Engineer with a Bachelors in Electronics Engineering from Southampton University and a Masters in Software Engineering from Oxford University.