More than 69 million people suffer traumatic brain injury every year. Many of them lose the ability to speak, type or do gestures. While several invasive treatment options are available for decoding their brain activity and helping them communicate with their loved ones, they require extensive brain surgery. Brain surgery carries risks and some are not even approved by the FDA.
To tackle this issue, researchers at Meta developed an AI to decode speech non-invasively in just 3 seconds of brain activity.
Let’s know more about this ongoing AI research in detail.
AI to decode speech from brain activity
The AI model developed at Meta can accurately decode words and sentences from brain waves. Using only a small amount of brain wave data, the AI guesses the correct answer up to 73% of the time. As it doesn’t require opening a patient’s skull, this non-invasive therapy is safer, provides a scalable solution and is beneficial to most people.
AI development, training and testing
Neuroscientists have been persistently working on decoding speech from neural signals non-invasively for a long time. However, non-invasive approaches are challenging because the recordings are noisy and can vary greatly between sessions and individuals for several reasons, including differences among people’s brains and where the sensors are located.
To address these challenges, researchers at Meta created a Deep Learning model, trained it with contrastive learning and then used it to maximise the alignment of non-invasive brain signals and speech sounds. They leveraged Wave2Vec, an open-source self-supervised learning model developed by their FAIR Team in 2020, approach to identify the complex representations in the brains of volunteers who listen to audiobooks. They even designed a new subject embedding layer, which is learned end-to-end to align all brain recordings in a common space.
Meta researchers focused mainly on electroencephalogram (EEG) and magnetoencephalography (MEG) technology. They utilised four open-source datasets from academic institutions, leveraging more than 150 hours of recording of 169 healthy volunteers listening in English and Dutch.
How does the AI Work?
The system performs zero-shot classification and determines the voice clip heard by the person from a large pool of voice clips. Then, the algorithm infers the words the person has probably heard in limited seconds of brain activity. It’s an interesting step as it shows AI can successfully learn to decode noisy and variable non-invasive recordings when speech is perceived.
It’s important to note that only 73% top-10 accuracy from a limited set of words can be detected. In practice, their approach benefits from pulling large amounts of heterogeneous data, which could, in principle, help improve the decoding of small datasets.
The need for non-invasive brain treatments
Connecting with your loved ones who’ve suffered from brain injury and being able to know what they’re thinking and feeling is a fulfilment of a dream. While existing brain-recording techniques provide unparalleled levels of clarity, they rely heavily on invasive practices. They require extensive surgery, a luxurious option for most people, which is highly risky.
Brain surgeries aren’t even FDA-approved in several cases. Even companies like Elon Musk’s Neuralink haven’t got FDA approval to be operated on humans. Therefore, many companies are now developing non-invasive alternatives that don’t require surgery and provide scalable solutions. Researchers are even working to make this solution affordable for the masses.
Is this AI the future of modern BCI technology?
While the AI model’s scope and capabilities remain limited, the researchers are optimistic about the future of brain-to-speech technology. However, detecting a small vocabulary is simply insufficient to permit effective conversation. “With language, that’s not going to cut it if we want to scale to practical use, because language is infinite.” – says Jonathan Brennan, a linguist at the University of Michigan in Ann Arbor.
For the AI to be a useful communication tool, scientists must work on how to interpret what the patients intend to say from brain activity, such as signs of hunger, discomfort, or a simple “yes” or “no.”
