iCONNECT

Reading brain signals for decoding speech With elderly people set to form an ever-greater proportion of the population in future, more of us will live with the consequences of diseases that may paralyse some muscles and lead to function loss. Professor Nick Ramsey tells us about his team’s work in developing a braincomputer interface designed to restore function and help paralysed people communicate.

A stroke in the brain stem is one of the main causes of total paralysis, damaging the connections between the brain and the muscles, and so impairing an individual’s ability to communicate. A second major cause is amyotrophic lateral sclerosis (ALS), a neurological disease that affects motor neurons, nerve cells which control voluntary muscle movement. “The signals are intact in the brain, and it generates the impulses, but basically the wires to the body are no longer working,” explains Nick Ramsey, a Neuroscience Professor at the Brain Center of the University Medical Center of Utrecht. As the Principal Investigator of the iCONNECT project, Professor Ramsey is working to develop a brain-computer interface (BCI) that helps paralysed people communicate, building on earlier research into the brain.

“We have been working with epileptic patients, who have electrodes implanted for their diagnosis. This gives us the opportunity to pursue basic research into how the brain works,” he outlines.

Brain signals This research led to important insights into how to interpret the brain’s signals, from which the idea of working with implants to decode inner speech was developed.

The foundation of this work is a detailed understanding of how signals are transmitted between the brain and muscles. “There are many muscles in your body, and they are all stimulated by a particular part of the brain, the primary motor cortex. That’s where the neurons reside, that get the pulses to the muscles,” says Professor Ramsey. Different parts of the body are organised in an orderly fashion, and their movements can be related to signals from specific parts of the brain.

“The cortical homunculus starts in the middle at the top of the head, and it goes to either side of the body – where the left part of your brain is connected to the right side of your body, and the other way round,” continues Professor Ramsey. “If we look at one side of the sensorimotor cortex, we can delineate which part of the brain maps onto which part of the body.”

22

Implantation of the Utrecht Neural Prosthesis in a Locked in patient. (www.neuroprosthesis.eu)

Researchers in the iConnect project have produced evidence that the movement of different muscles in the face leads to different patterns on the cortex. So for example if an individual purses their lips or clenches their jaws, then researchers monitoring their brain activity would see a pattern, where very small patches of

cortex become active. “That pattern allows us to identify what kind of movement you’re making, we can even identify different spoken letters such as ‘p’ or ‘ah’. We’ve also proved that if you cannot make a movement – but try to – then you still get the same patterns on the cortex. This supports the idea that in cases of paralysis the brain is intact and the pulses are still generated, but they don’t actually arrive at the muscles,” says Professor Ramsey. This is central to the project’s work in developing an intracranial BCI, designed for use in the home. “We decided to first try and accomplish something relatively simple, but which really helps patients who are locked-in, who are unable to communicate,” outlines Professor Ramsey.

The long-term, ambitious goal in this research is to interpret brain signals so accurately that it becomes possible to develop implants that translate attempted speech to a speech computer in real-time, and implants that make muscle movements possible again. “We aim to offer a system to people with locked-in syndrome that will help them to communicate again,” explains Professor Ramsey.

A core part of this work centres around developing an implant that will record brain signals, interpret them, and send the

On the left the Utrecht Neural Prosthesis is shown.

On the top right a 7 Tesla functional MRI scan of the 5 fingers of the right hand (thumb/orange to little finger/red). On the bottom right the electrode grids for the next generation BCI for decoding speech and gestures.

EU Research

The iCONNECT team

iCONNECT Intracranial COnnection with Neural Networks for Enabling Communication in Total paralysis Project Objectives

interpreted signals to another small computer.

The second computer will then instruct the muscles to make particular movements, so that motor functionality can be restored in paralysed people.

The system itself is relatively basic at this stage however, enabling users to generate a click when they scroll through a drop-down menu and to communicate in that way.

Researchers have achieved a high level of reliability, which Professor Ramsey says is very important in terms of moving the technology into the homes of people who need it. “Our success in achieving this constituted a major breakthrough for BCI, since for the first time a locked-in person could use the implant at home any time it is needed, without requiring expert help.” (www.neuroprosthesis.eu)

Aging population The backdrop to this research is Europe’s changing demographic profile, with the elderly set to account for a greater proportion of the population over the coming years. The risk of suffering a brain haemorrhage or stroke increases dramatically with age, and with life expectancy increasing, Professor Ramsey believes there will be significant demand for this technology in future. “If you suffer a stroke at the age of 70, you may still live for another 20 years, so we want to develop technology to help people live in a dignified way,” he says. There is also the possibility of helping people with less severe disabilities, helping widen the market for this technology, which is an important consideration for potential commercial partners looking to develop it further. “We

We’ve also proved that if you cannot make the movement – but try to – then you still get the same patterns on the cortex. This supports the idea that in cases of paralysis the brain is intact and the pulses are still generated, but they don’t actually arrive at the muscles.

This provides the foundation for further research into brain signals, opening up the possibility of adding functionality in future.

The existing implant has four amplifiers, so it can record and transmit data from four electrodes on the brain, now Professor Ramsey is planning a study which he hopes will lead to deeper insights into brain signals. “As we get more electrodes, they get smaller and we can put them closer together, so we can look more deeply into the features of brain signals.

It’s a constant process of improving the quality of the de-coding, by building a deeper understanding of brain signals,” he outlines. A key goal in the near future will be to achieve what is called point-and-click functionality.

“Currently users can generate a click, but they have to wait until the icon or letter of interest lights up, so it’s quite slow. If we can manage point-and-click, where a user can move the mouse by brain signals and generate a click, then that will be a major upgrade,” explains Professor Ramsey.

www.euresearcher.com

must get better at decoding the brain’s signals if we are to also help people with less severe disabilities, which will expand the functionrestoring capabilities of this technology. The more sophisticated the technology, the more people you can help,” says Professor Ramsey.

This is a new area of research, as we are only just discovering how a person can be taught to use their brain signals to control a computer, while it also raises ethical questions. A patient at risk of getting locked-in syndrome may well want to make their own decisions about their future, based on their own quality of life, something of which Professor Ramsey is well aware. “We are aware of the need for an ethical framework around this kind of work,” he says. Professor Ramsey publishes work widely in medical journals, aiming to heighten awareness of the potential of BCI systems.

“We publish as much as we can in medical journals, so that we can make neurologists aware that these patients actually report a good quality of life,” he continues.

Many people suffer from partial or full loss of control over their body due to stroke, disease or trauma, and this will increase as elderly people account for a greater proportion of the population. With both duration and quality of life beyond 60 increasing in the western world, more and more people will suffer from the consequences of function loss, and will stand to benefit from the development of restorative technology. iCONNECT aims to give severely paralyzed people the means to communicate by merely imagining themselves talking or making hand gestures. Imagining specific movements generates spatiotemporal patterns of neuronal activity in the brain which Professor Ramsey explains.

Project Funding

Funded by an ERC Advanced Grant Systems and communication engineering.

Contact Details

Project Coordinator, Professor Nick F. Ramsey Brain Center Rudolf Magnus University Medical Center Utrecht Room G03 1.24 Huispostnummer G03 1.24 PO Box 85500 3508 GA UTRECHT T: +31 (0)88 755 6862 E: n.f.ramsey@umcutrecht.nl W: http://www.nick-ramsey.eu

Professor Nick Ramsey

Nick Ramsey is full professor in cognitive neuroscience at UMC Utrecht’s department of neurology and neurosurgery, a position he has held since 2007. His primary goal is to acquire and translate neuro-scientific insights to patients with neurological and psychiatric disorders, with a focus on braincomputer interfacing.

23