9 minute read
Sleep Stage Scoring with Artificial Intelligence
By Matteo Cesari, PhD and Alexander Wachter, MSc
Sleep stage scoring is a fundamental task when objectively evaluating sleep with polysomnography (PSG), the gold standard method for sleep assessment. The current rules for sleep stage scoring are defined in the Manual for Scoring Sleep and Related Events, which is regularly updated by the American Academy of Sleep Medicine (AASM). Using these rules, sleep experts and technicians manually score the PSG recordings, labeling each 30-second period of sleep (also called an “epoch”) as wakefulness, rapid eye movement (REM) sleep, or one of the three non-REM sleep stages. Manual sleep stage scoring is time-consuming and can be inconsistent because the rules for scoring sleep stages are prone to subjective interpretation. When a PSG recording is given to two different technicians, they tend to agree on only 8590% of the epochs. The agreement decreases significantly for recordings of patients with neurodegenerative diseases.
Automatic methods would be highly beneficial in this field. Algorithms are faster and provide consistent and objective scoring, overcoming both problems of time consumption and subjective interpretation of PSG recordings. The first attempts to automatically score sleep stages began in the 1960s when simple rule-based algorithms were proposed. Such algorithms consisted in a series of human-defined rules to try to mimic manual scoring. However, due to their simplistic approach and the many exceptions that could not be foreseen by the rules, the algorithms did not perform as well as human scoring.
Artificial intelligence (AI) is a branch of computer science in which, to complete the required task,
the computer captures the patterns between the data and the desired output on its own without explicitly defined rules from a programmer. In the context of sleep stage scoring, the computers are given a set of thousands of sleep epochs previously scored by humans to learn the relationship between the data and the expected sleep stage. Once the learning is complete, the AI algorithms are tested on new unseen epochs and compared with human scoring to understand whether the algorithm performs as well as humans. In recent years, different research groups have proposed new AI algorithms, which reached the same accuracy as human experts in scoring sleep stages. These achievements are due to the increased computational power of computers as well as to the availability of large datasets, which allow computers to learn even the most difficult and rare patterns.
The results recently achieved by AI algorithms are promising in the sense of making AI-based sleep scoring a routine process in sleep labs around the world. This would allow sleep labs to evaluate PSGs faster and make it possible to record more patients and/or to reduce the costs related to PSGs as well as increase the consistency between different sleep labs, making evaluation more standardized. Thanks to the increasing development of more portable at-home PSG systems, AI-based sleep stage scoring has the potential to record even more sleep studies and make waiting times for PSGs significantly shorter.
In addition to scoring PSGs, research studies show that AI-based sleep stage scoring might provide new information. AI algorithms not only provide the sleep stage for each sleep epoch but also estimate the probability of such stage, which is useful to better understand the transition between different stages and the stability of sleep structure. Research shows that the instability measures derived from these probabilities might be useful for improved diagnosis of narcolepsy type 1. Furthermore, since sleep is a continuous and dynamic process, automatic AI-based scoring could be adapted to score sleep epochs shorter than 30 seconds. This is better suited to capture the underlying information rather than simplifying it to 30-second epochs, which is a rudiment dating from the time sleep was scored on paper.
Recent evidence shows that AI-based sleep stage scoring could potentially be used in clinics and, in the not so distant future, to improve the diagnosis of sleep disorders. The integration of AI in sleep medicine, however, requires clear guidelines to build trust in these automatic procedures. The AASM is actively working in this direction with a dedicated taskforce. To certify the reliability of AI-based sleep stage scoring algorithms, the AASM is developing a certification program in which AI algorithms for sleep stage scoring will be tested on unseen recordings owned by AASM and scored by several scorers. Based on the agreement with the human scorings, AI algorithms will receive a certification ensuring their reliability.
The time for moving from manual to AI-based sleep stage scoring is approaching, and clinics and sleep centers need to be prepared for this shift, which will reshape their organizational structure as well as the approach with patients.
*Citations available on healthiersleepmag.com Matteo Cesari, PhD
Alexander Wachter, MSc Matteo Cesari, PhD has been working in the field of sleep medicine & research since 2016. Dr. Cesari is currently Postdoc Fellow at the Sleep Disorders Unit, Department of Neurology, Medical University of Innsbruck in Austria.
Alexander Wachter, MSc has been in the field of computer science since 2020. He is currently PhD student at the Sleep Disorders Unit, Department of Neurology, Medical University of Innsbruck in Austria.
The CG Fund
In his memory, World Sleep Foundation, a nonprofit 501c(3) organization, created an endowment in honor of Dr. Guilleminault. The CG Fund awards travel grants to young investigators in sleep medicine and research to encourage and enable their participation in the scientific sleep community.
CG Award Applications
Applications for the CG Award are accepted year-round and reviewed three times per year. The deadlines are:
April 30 | August 31 | December 31 Christian Guilleminault “CG” 1938 – 2019
Dr. Christian Guilleminault (known as “CG” among colleagues) devoted his career to the development and advancement of sleep medicine and research. His groundbreaking research in the areas of sleep apnea, pediatric sleep disorders, and narcolepsy made him a leader in the field of sleep medicine and research. Throughout his career, Dr. Guilleminault mentored hundreds of physicians and scientists.
Get Involved!
• Contribute to the CG Fund • Share the CG Award application with colleagues
Scan the QR code with your phone camera or visit worldsleep.info/yi for more information.
Right Now in Sleep Science
Detecting Parkinson's Disease through Sleep
Researchers in the United States have discovered a way to use artificial intelligence (AI) to evaluate Parkinson's disease (PD), a neurological disease that affects more than 10 million people worldwide.
PD is a progressive disorder that does not have a specific diagnostic test. PD is typically diagnosed based on patient reports of symptoms like tremors in the hand muscles or stiff arms or legs. However, these symptoms usually do not show up until several years after the disease has started, leading to late diagnosis and missed opportunities for early treatment.
There is a link between PD and breathing, though, and sleep breathing disorders often present years before PD symptoms. In the study, researchers used an AI-based system to look for abnormal nighttime breathing patterns in a set of data with over 7,500 people and nearly 12,000 nights. The breathing signals were collected using either a wearable device (in this case, a breathing belt worn on the person’s chest or abdomen) or nearable device (a bedside radio signal transmitter). Researchers found that for patients using the wearable device for one night, the AI system was about 80% accurate in identifying PD. For patients who used the nearable device for several nights, the accuracy was even higher.
The researchers from the study are hoping the AI-based system can lead to earlier diagnosis and treatment of PD. Using wearable or nearable technology could also be easier and less expensive for patients, both in diagnosing the disease and tracking its progression.
References Yang Y, Yuan Y, Zhang G, et al. Artificial intelligence-enabled detection and assessment of Parkinson’s disease using nocturnal breathing signals. Nature Medicine. 2022; 28: 2207–2215 doi: https://doi.org/10.1038/ s41591-022-01932-x
https://www.nature.com/articles/s41591-022-01932-x#authorinformation
These days you can use technology to track almost anything in your life – including your sleep! You may have noticed that a watch or device that you already own records sleep data, or maybe you’ve purchased a specific tracker just for that purpose. Over the past few years these devices have become very popular.
Whether you choose to use a watch, a bracelet, your phone, or other device, what do you do with the data once you get it? Can it help you improve your overall health? We spoke with sleep expert Massimiliano de Zambotti, PhD about how these trackers work and what to do with the information they provide. According to Dr. de Zambotti, sleep trackers can record data such as sleep timings, how long you sleep, and nighttime awakenings. The newer generation of sleep trackers can also measure heart rate and its variability, blood oxygen levels, breathing patterns, and even stages of sleep. The data from the device sensors are processed by the manufacturer’s algorithms, then the data are combined and presented to the user through an app or website.
For a tracker to be most helpful, it must record accurate and reliable data and it should also provide the right contextualization and information to interpret it. Dr. de Zambotti says that at this time, consumer devices can’t be used to diagnose specific sleep disorders, and it’s still
Dealing with the Data
By Rosei Skipper
unclear how helpful the data can be from a clinical standpoint. The devices can help individuals be more aware of their sleep and may even help users to improve their sleep habits over time. However, while reaching the magic 10,000 steps per day for activity is under one’s control, achieving a specific sleep target is not so straightforward.
Some devices generate a “nightly sleep score” or a “grade” of the previous night’s sleep. Dr. de Zambotti explained that the goal of the sleep score is to take the data collected overnight and turn it into a number that reflects the quality of sleep of the user. It is still unclear, however, what this number truly represents. For example, you might wake up feeling refreshed and energetic, but your sleep score is lower than usual. Because scientists don’t have a full understanding of what goes into the sleep scores for different devices, Dr. de Zambotti said to pay more attention to how you feel and to other standard values these devices provide that have more actionable insights. Dr. de Zambotti suggests focusing on objective numbers like what time you go to bed and wake up.
While some people benefit from tracking their sleep over weeks or months, others might find themselves getting overly invested with the information. Some users can get competitive about improving their sleep score or even feel anxiety about the data. It’s important for each person to note whether they feel a tracker and the data it generates are helpful to them as an individual.
The bottom line? If you think a sleep tracker might be helpful to you, try it out! There are many options, so make sure to research various devices and pick one that matches your individual lifestyle and budget. And as always if you have concerns about your sleep or health, be sure to contact your health care team.
Rosei Skipper, MD completed her Psychiatry residency and Child fellowship at the Mayo Clinic in Rochester, MN. She is currently pursuing further training in psychoanalytic therapy.
Massimiliano de Zambotti, PhD is a sleep neuroscientist serving as the Lead of the Translational Sleep Technology Unit, Human Sleep Research Program at SRI International in Menlo Park, California. He is also the Cofounder and Chief Scientific Officer of Lisa Health Inc.
