Media release

From: Springer Nature

Neuroscience: Decoding brain activity to spell sentences

A device capable of decoding brain activity in a participant with speech paralysis, as they silently attempt to spell out words phonetically to create full sentences, is reported in Nature Communications. The findings highlight the potential of a silently controlled speech neuroprosthesis to generate sentences through a spelling-based approach.

Neuroprostheses are devices that replace lost nervous system function, and have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Previous research has shown that a neuroprosthetic system in a participant with speech paralysis can decode up to 50 words. However, this system was limited to a specific vocabulary and the participant had to attempt to speak the words out loud, which required significant effort given their paralysis.

Edward Chang and colleagues designed a neuroprosthesis capable of translating brain activity into single letters to spell out full sentences in real time, and demonstrated its use in a participant who suffered from limited communication because of severe vocal and limb paralysis. The authors expanded the previous approach to a larger vocabulary by designing their system to decode brain activity associated with the phonetic alphabet. In tests, the device was able to decode the brain activity of the participant as they attempted to silently speak each letter phonetically to produce sentences from a 1,152-word vocabulary at a speed of 29.4 characters per minute, and an average character error rate of 6.13%. In further experiments, the authors found that the approach generalized to large vocabularies containing over 9,000 words, averaging a 8.23% error rate.

The results highlight the potential of silently controlled speech neuroprostheses to generate sentences through a spelling-based approach using phonetic code words. Further work is required to demonstrate if this approach is reproducible in more participants.