Researchers have unveiled a new brain-computer interface developed at the University of California, Davis, that translates brain signals into speech with 97% accuracy. This interface offers hope to patients with neurological disorders that prevent them from speaking by knowing what they want to say and conveying it to those around them.
In a study published in the New England Journal of Medicine on Aug. 14, researchers implanted sensors in the brain of a man with severe speech impairment due to amyotrophic lateral sclerosis. The man was able to communicate what he intended to say within minutes of activating the system.
amyotrophic lateral sclerosis
Amyotrophic lateral sclerosis, also known as Lou Gehrig’s disease, affects the nerve cells that control movement throughout the body. The disease gradually causes people to lose the ability to stand, walk, and use their hands. It can also cause people to lose control of the muscles used for speech, leading to a loss of the ability to speak clearly.
This new technology has been developed to give the ability to communicate to people who cannot speak due to paralysis or neurological conditions such as amyotrophic lateral sclerosis. The new technology can interpret brain signals when the user tries to speak and convert them into text that is spoken aloud by a computer.
“Our brain-computer interface technology helped a paralyzed man communicate with friends, family and caregivers, and our study demonstrates the most accurate speech translation device ever,” said David Brandman, a UC Davis neurosurgeon, co-principal investigator and co-author of the study, assistant professor in the Department of Neurological Surgery at UC Davis and co-director of the Brain-Prosthetics Laboratory.
The new device breaks the communication barrier
When a person tries to speak, the new interface converts brain activity into text that appears on a computer screen, and the computer can then read the text aloud.
Casey Harrell, a 45-year-old man with amyotrophic lateral sclerosis, was participating in the BrainGate clinical trial to develop the system. At the time of enrollment, Harrell had weakness in his arms and legs (tetraplegia), his speech was extremely difficult to understand (dysarthria) and he needed help from others to interpret his speech.
In July 2023, Brandman implanted the experimental device, implanting four microarrays of electrodes in the left central gyrus, an area of the brain responsible for coordinating speech.
“We’re actually detecting them trying to move their muscles and speak,” explained neuroscientist Sergei Staviski, an assistant professor in the Department of Neurosurgery, co-director of the UCLA Neuroprosthetics Laboratory and co-principal investigator on the study. “We’re recording from the part of the brain that’s trying to send those commands to the muscles, listening to those signals and translating those brain activity patterns into phonemes—like syllables—and then into the words they’re trying to say.”
Faster training, better results
Despite recent advances in brain-computer interface technology, efforts to enable communication have been slow and error-prone. This is because machine learning programs that interpret brain signals require a lot of time and data to work.
“Previous brain-computer interface systems had frequent word errors, which made it difficult for the user to consistently be understood, and this was a barrier to communication,” Brandman explained, according to EurekAlert. “Our goal was to develop a system that would enable a person to be understood when they wanted to speak.”
Harrell used the system in situations that required both guided and spontaneous conversations. In both cases, speech decoding occurred in real time, with continuous updates to the system to maintain its accuracy.
The disjointed words were displayed on a screen, and the words were spoken in a voice that sounded surprisingly similar to Harrell’s before he developed ALS. The voice was created using software trained on existing voice samples of his pre-ALS voice.
The system took 30 minutes to achieve 99.6% accuracy with a vocabulary of 50 words in the first training session of audio data.
“When we first tried the system, he cried with joy when the words he was trying to say appeared on the screen correctly,” Stavisky said. “We all cried.”
In the second session, the potential vocabulary expanded to 125,000 words. With just an additional hour of training data, the system achieved 90.2% accuracy with this much-increased vocabulary. After collecting more data, the system maintained 97.5% accuracy.
“At this point, we can decode what Casey is trying to say about 97 percent of the time, which is better than many commercial applications available that try to interpret a person’s voice,” Brandman said. “This technology will be transformative because it offers hope to people who want to talk but can’t. I hope that in the future, this brain-computer interface technology will help patients talk to their families and friends.”