Paralysed woman 'speaks' after 18 years through avatar

In a groundbreaking advancement, a woman who has been severely paralysed for over 18 years due to a brainstem stroke has successfully communicated through a digital avatar, utilising the brain-computer interface (BCI) technology.

This remarkable breakthrough offers renewed hope for individuals who have lost their ability to speak due to conditions such as strokes and amyotrophic lateral sclerosis (ALS).

Translating brain signals into speech and expressions

Traditionally, individuals who have lost the ability to communicate verbally have relied on laborious speech synthesisers operated through eye tracking or limited facial movements. However, this new technology, developed by researchers at the University of California, San Francisco (UCSF), employs minuscule electrodes implanted on the brain's surface to capture electrical signals from the speech and facial expression control regions. These signals are then instantaneously translated into speech and facial expressions by a digital avatar.

"Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others," Professor Edward Chang, lead researcher at UCSF, told The Guardian.

This achievement brings us significantly closer to providing a tangible solution for patients who have struggled to communicate naturally, he added.

The patient at the heart of this breakthrough is a 47-year-old woman named Ann, who has been living with profound paralysis following a brainstem stroke for nearly two decades. Ann's communication has primarily relied on technology that enables her to select letters painstakingly, limiting her speed to just 14 words per minute.

Now, with the introduction of this new technology, Ann hopes to resume work as a counsellor.

Unveiling the technology

The team implanted a paper-thin rectangle of 253 electrodes onto the surface of Ann's brain, over a region critical for speech. The electrodes intercept the brain signals that would control the muscles in her tongue, jaw, larynx, and face, had the stroke not occurred.

Following the implantation, Ann collaborated with the researchers to train an AI algorithm to recognise her unique brain signals for various speech sounds by repeatedly vocalising different phrases.

The AI learned to distinguish 39 distinct sounds, and a language model similar to ChatGPT was used to transform these signals into coherent sentences.

The avatar's voice was tailored to match Ann's pre-injury speech, based on a recording from her wedding day.

Progress and challenges

While the technology is groundbreaking, it is not without its imperfections. In tests encompassing more than 500 phrases, words were decoded inaccurately 28% of the time.

Additionally, the brain-to-text rate reached 78 words per minute, compared to the 110-150 words spoken in natural conversation.

Nonetheless, experts believe that the current levels of accuracy, speed, and sophistication mark a significant advancement, making the technology practically applicable for patients.

A path towards independence

Scientists and medical experts are optimistic about the potential of BCI technology, emphasising its potential to grant individuals the autonomy to control computers and devices independently. The prospect of a wireless version of the BCI implanted under the skull presents a crucial future step.

"Giving people the ability to freely control their own computers and phones with this technology would have profound effects on their independence and social interactions," Dr David Moses, co-author of the research and an assistant professor in neurological surgery at UCSF, told The Guardian.

In essence, this groundbreaking achievement not only brings hope to those who have lost their ability to speak but also introduces a new era of communication possibilities through innovative technology.