Media release

From: Springer Nature

Brain–computer interfaces (BCIs) that are capable of decoding brain activity into speech faster, more accurately and covering larger vocabulary than existing technologies are reported in two papers published in Nature. The findings demonstrate advances in the development of technology that aims to restore communication to people with severe paralysis.

People with neurological disorders, including brainstem stroke or amyotrophic lateral sclerosis, often face speech loss due to paralysis of muscles. Previous studies have shown that it is possible to decode speech from the brain activities of a person with paralysis, but only in the form of text and with limited speed, accuracy and vocabulary.

Francis Willett and colleagues developed a BCI that collects the neural activity of single cells with an array of fine electrodes inserted into the brain, and trained an artificial neural network to decode intended vocalizations. With the help of the device, a patient with amyotrophic lateral sclerosis was able to communicate at an average rate of 62 words per minute, which is 3.4 times as fast as the previous record for a similar device and moves closer to the speed of natural conversation (around 160 words per minute). The BCI achieved a 9.1% word error rate on a 50-word vocabulary, which is 2.7 times fewer errors than the previous state-of-the-art speech BCI from 2021. A 23.8% word error rate was achieved on a 125,000-word vocabulary.

In a separate study, Edward Chang and colleagues developed a BCI based on a different method for accessing brain activity, using nonpentrating electrodes that sit on the surface of the brain and detect the activity of many cells across sites over the entire speech cortex. This BCI decodes brain signals to generate three outputs simultaneously: text, audible speech and a speaking avatar. The researchers trained a deep-learning model to decipher neural data collected from a patient with severe paralysis, caused by a brainstem stroke, as she attempted to silently speak full sentences. Brain-to-text translation generated a median rate of 78 words per minute, which is 4.3 times as fast as the previous record and moves even closer to the speed of natural conversation. The BCI achieved a 4.9% word error rate when decoding sentences from a 50-phrase set, which is 5 times fewer errors than the previous state-of-the-art speech BCI. A 25% word error rate was achieved when real-time decoding sentences with a 1,000+ word vocabulary, and offline simulations showed a 28% word error rate using a vocabulary with over 39,000 words. Brain signals were also translated directly into intelligible synthesized speech sounds which untrained listeners could understand, with a word error rate of 28% for a set of 529 phrases, and personalized to the participant’s pre-injury voice. The BCI also decoded neural activity into an avatar's facial movements during speaking, as well as non-verbal expressions. Stable, high-performance decoding was demonstrated over months. Overall, this multimodal BCI provides more possibilities for people with paralysis to communicate in a more naturalistic and expressive way.

Nick Ramsey and Nathan Crone write in an accompanying News & Views that the two BCIs “represent a great advance in neuroscientific and neuroengineering research, and show great promise in alleviating the suffering of individuals who have lost their voice as a result of paralysing neurological injuries and diseases.” They note that further work is required to allow for more widespread use.