Feb 29th, 2024 - By Sam Hosovsky* (Contributor #1), Oliver Shetler (C#2), Luke Turner (C#3), Caila Kinnaird (C#4)
Once, Noah200 dreamed he was a robin, a robin flickering and singing about, happy with himself and doing as he pleased. He didn't know that he was Noah200.
Suddenly he woke up and there he was, solid and unmistakable Noah200. But he didn't know if he was Noah200 who had dreamt he was a robin, or a robin dreaming that he was Noah200.
Immersive technologies can elicit sensations that rival those of our dreams, deceiving our perceptions of the world and of ourselves. Our senses, after all, are not foolproof. Virtually all of us have had our perception artificially altered—be it with perfumes, coffee, digital displays, or music. Endogenously, consider that 1% of us live with hallucinations or delusions due to schizophrenia (Casarella, 2022.
The difference is that we have never altered our perceptions with such fidelity and conviction as with Virtual Reality (”VR”).
The fidelity is achieved by connecting millions of points in 3D space into triangles, storing them in half-edge data structures, and continuously projecting them onto polygon meshes we recognize, along with properties defining how they interact with light, sound, and so on.
The conviction, on the other hand, relies on the believable algebraic transformation of these structures in response to time, the user’s motion, and interaction with the space.
<aside> 🌐 Harnessing VR, people who are immobilized in the physical world can find joy and fulfillment in the virtual one.
</aside>
This sentiment is especially shared among those with disabilities who make up a whopping 50% of SecondLife users—one of the most mature social virtual worlds (French, 2017).
Some of the most unfortunate are those whose lively minds are imprisoned by their own paralyzed bodies. Hearing, seeing, thinking but unable to move or express themselves.
Primitive tools like sip and puff switches allow them to communicate slowly, paired with a begrudging but valiant effort from their supportive counterparts. The space between a letter-covered plexiglass and the arduous breath patterns individuals must endure is filled with a motionless, silent, yet deafening frustration.
Like the SecondLife users, many people with paralysis across the world are opting for VR solutions: ones that maximize freedom of expression, rehabilitation, and positive clinical outcomes. While they stand to greatly benefit from VR, their control of it becomes near impossible. Fortunately, another technology, the Brain-Computer Interface (”BCI”), is stepping up to the challenge.
Implanted BCIs can decode their intended movements and speech directly from the associated brain areas before the paralysis nullifies them. After all, on a certain level, our actions are but an electrical signal.
Coupling VR with BCI even those with severe paralysis can become the robins, mermaids, scientists, dancers, or whatever forms they’ve dreamt of being. Let’s embark on a journey to progressively evolve this seemingly impossible idea into a clinically feasible, practical solution.
The idea of embodying our minds within personalized avatars and seamlessly immersing ourselves in virtual worlds has long been a speculation of science fiction. It has been extensively explored through sci-fi cinema spectacles, video games, and philosophy, which has set a precedent for an uncertain mind-controlled future. Increasingly, however, aspects of this speculation are no longer confined to the realms of fantasy.
As of the year 2024, the performance of Cortical Motor Neuroprosthetics (”Motor BCIs”) is beyond the tipping point for practical applications. In short, by listening in on the rich activity of the motor cortex they can interpret the mind's commands for the body's musculature.
Significant advances in the ability of these Motor BCIs to restore motor function have gathered support for three critical hypotheses:
For most people, Motor BCIs can improve their connection with technology. For people with severe communication and mobility difficulties (”Users”), however, they represent the hope to reclaim their agency, expression, independence, and participation in society.
Academic labs have demonstrated the technical feasibility of BCIs, but clinical adoption lies in the hands of private firms. As the scientific endeavor tips over into an engineering one, the world watches closely.
As customary in the deep technology sector, many leading scientists have launched commercial ventures of their own, commercializing their academic work in the shape of: Neuralink, Paradromics, Synchron, Precision Neuroscience, ONWARD, CorTec, Blackrock Neurotech, NeuroXess, and INBRAIN Neuroelectronics.
This handful of high-performance Motor BCI startups share over $2 billion in funding without a single product approved for use in the clinical market by regulators such as the Food and Drug Administration (”FDA”).
Patients, doctors, and families all wait on bated breath for the cut of the ribbon. Some have been waiting decades. Some never lived to see the ribbon.
<aside> 🌜 Bringing Motor BCIs to market is a challenge akin to building a base on the moon.
</aside>
Nonetheless, given these hypotheses, the industry should formulate applications to maximize User value while remaining commercially viable. Yet, the range of formulated applications that motivate this development is underwhelming. Each startup boasts technology specs while neglecting to reveal how exactly their Users are to benefit. How can such multi-disciplinary efforts yield a narrow-minded outlook on patient outcomes?
A clinical application most favored by the key startups is the ability to control a computer cursor, granting the User some independence in performing digital activities of daily living. The startups’ longer-term aspirations aim towards the command over an exoskeleton, a robot, or the User's own body by bypassing the region affected by paralysis.
While the former application disappointingly shrinks rich decoded movement into mere 2D cursor control, the latter will long be unavailable outside the lab and carry a significant price tag.
However, there is another valuable, affordable, and feasible, yet underrepresented, application for Motor BCIs—that of emulating the User’s attempted movement and speech on an expressive virtual body ("Avatar"), embodied inside VR. Commercially viable on its own, such an application is also an important stepping stone for longer-term robotic aspirations, serving as a simulator for diverse everyday situations.
What you are about to read is the world’s first practical proposal of a commercial Motor BCI to VR system, which includes a rigorous evaluation of User needs, modern Motor BCI capabilities, and their joint realization in VR. This system is called “uCat” and its prototype is revealed towards the end.
Severe paralysis can affect anyone, at any time, regardless of age, gender, or socioeconomic status, rendering you unable to move and speak all the while preserving your cognition. Etiologies among the most prevalent are Spinal Cord Injury ("SCI"), Stroke, Amyotrophic Lateral Sclerosis ("ALS"), Traumatic Brain Injury, Multiple Sclerosis, Cerebral Palsy, and Muscular Dystrophy.
Despite being a diverse population, people with severe paralysis share one common goal: to express themselves again and participate equally in society as autonomous individuals. This yearning unites their community as it does ours; expression sits at the foundation of all human experience.
Figure 1; Gleason, 2014: A quote from a former NFL player with ALS. Reprinted from public materials from the Team Gleason Foundation.
Whether someone's expression was snatched from them in an instant or if it slowly degraded over time, there still remains no cure for paralysis.
Motor BCIs are stepping up to the challenge by finding ways to extract bodily commands and harness them to control devices before they are nullified by paralysis. The more they can tell us, the more we can restore. Before exploring their achievements, you should get familiar with the problems they are attempting to address.
Among those most impaired are individuals affected by Locked-In Syndrome (”LIS”), characterized by aphonia (loss of voice) and quadriplegia, but preserved cognition.
Such is the fate of those diagnosed with ALS, a no-cure progressive degeneration of motor neurons, responsible for 55% of LIS cases, or those affected by a sudden brainstem stroke, causing 25% of LIS cases (Pels et al., 2019).
In total, roughly 1 million individuals are destined to be locked inside their own bodies at any given time (Mehta et al., 2017).
Characterized by requiring electric wheelchairs with postural support and speaking aids (referred to as Augmentative & Alternative Communication ”AAC”), roughly 21 million individuals seek to restore their independence, expression, and participation (WHO and UNICEF, 2022).
While the characteristics overlap, the etiologies are diverse. In addition to the aforementioned, another large population to share these characteristics are the survivors of high cervical (C1-C4) SCI (Shepard Center, n.a..
The prevalence of SCI significantly varies between countries. In the US, however, roughly 1 in 1000 people live with SCI (Singh et al., 2014). Nearly 1/3 of them, according to a large-population study by Amidei et al. (2022), with high cervical lesions.
Revisiting stroke: Roughly 101 million people (or 1 in 80 people) live with stroke (WSO, 2022). Approximately 5% of them suffer from severe dysarthria (speech paralysis) without aphasia (language incomprehension) (Mitchell et al., 2021), suggesting healthy cognition without vocalized expression.
While therapies vastly differ between conditions and severities, most individuals experience only slight improvements in mobility and expression—if any at all—and rely on attendant care to perform most activities of daily living.
Lead by the cost of attendant care (~$150k), the annual cost of care for severe paralysis is approximately $350k, including ~$54k incurred in lost wages (DeVivo et al., 2011, Obermann and Lyon, 2014, Oh et al., 2015):
Figure 2; uCat, 2023: Visualization of the costs of severe paralysis per individual per year (pro-rata) - Combining the findings of DeVivo et al. (2011), Obermann and Lyon (2014), Oh et al. (2015), charges were primarily gathered from SCI data (n=1000+) as ALS data included a smaller sample size. Ventilation was the only cost derived from ALS dataset. The analysis focused on the U.S. market, converting the purchasing power of the dollar to 2023, rounding to the nearest 1000th below $50k and 10,000th above 50k.
In the US, much of this cost is covered by the health insurance provided by the Centers for Medicare & Medicaid Services (”CMS”).
To consistently evaluate each claim, CMS classifies them against a standard set of codes. Each code is associated with a specific payment amount, which can be used by CMS to calculate reimbursements to Motor BCI providers.
Under the Health Insurance Portability and Accountability Act (HIPAA), The American Medical Association and CMS maintain and distribute the Healthcare Common Procedure Coding System (”HCPCS”) Level I (commonly referred to as “CPT”) and Level II codes, respectively (CMS, 2022).
<aside> 🔖 The alignment with CPT and HCPCS Level II codes grounds Motor BCIs in standard reimbursement schemes which is the only way to scale their (US) distribution to the populations of severely paralyzed.
</aside>
Owing to its novelty, many of the relevant codes for Motor BCIs have not yet been established, and it is up to the manufacturers to request new ones (e.g. NeuroPace Announces New Category I CPT Code).
Navigating the complex coding systems to uncover reimbursement codes relevant to Motor BCIs is beyond the scope. However, one can find many relevant codes by observing medical devices with somewhat similar characteristics.
<aside> 📡 HCPCS Codes: NeuroPace RNS
</aside>
<aside> 🖲️ HCPCS Codes: Medtronic DBS
</aside>
The only two fully implanted intracranial BCIs with chronic electrophysiological recordings in the market are NeuroPace's Responsive Neurostimulator ("RNS") and Medtronic's Percept PC Deep Brain Stimulator (”DBS”) systems. Although not designed for motor impairments, they each record, analyze, store, and visualize brain data to responsively trigger a custom stimulation protocol, preventing symptoms of epilepsy and Parkinson’s disease, respectively.
Figure 3; Oliveira et al., 2023: “Application of machine learning towards adaptive deep brain stimulation using closed-loop control. In the top panel, a list of challenges at different stages of DBS therapy implementation where ML methods can play an important role. The diagram represents a closed-loop feedback system for aDBS, based on LFPs sensing and electrophysiological biomarkers identification and interpretation.” Reprinted from Figure 1.
The first-generation Motor BCIs are not concerned with neurostimulation. Instead, they expect the User to control an external device. This outlook may change as approaches of reactivating one's own muscles (demonstrated early by Bouton et al., 2016 on hand-related tasks and recently by Lorach et al, 2023 on walking tasks) and restoring haptic percepts (Greenspoon et al., 2023; Valle et al., 2024) reach clinical maturity. Only recently, Herring et al., (2023) managed to combine some of these approaches into a proof of concept bidirectional Motor BCI bypassing the injury. I talk more about it in Other Electronic Applications.
Unfortunately, even the ‘external device’ options remain limited as Motor BCI arm prostheses (mounted on an electric wheelchair) are expected to cost roughly $100k (McGimpsey and Bradford, 2017) and exoskeletons a whopping $250k (Limakatso, 2023, well beyond any justifiable cost-effectiveness ratio. I further review the robotic Motor BCI applications in Converging VR and Robotics.
Myoelectric prosthetic hands, like the Ability Hand from Psyionic, are bringing the cost down to $20-30k which approaches the rates reimbursable with existing HCPCS codes. As before, these codes can act as a proxy for the more sophisticated Motor BCI robotics.
<aside> 🦾 HCPCS Codes: Myoelectric Hands
</aside>
Therefore, the prevailing Motor BCI application settles for control of a standard computer cursor to spell out letters or control a user interface. Here, the solution competes with AAC devices, such as eye-tracking, which offer Users similar functionality non-invasively. Again, their HCPCS codes can help approximate Motor BCI reimbursement options. For example, the Tobii Dynavox I-Series is the most popular eye-tracker among the ALS community and has an HCPCS code of E2510 (Tobii, n.a.
<aside> 🗣 **HCPCS Codes: AAC and Cursor**
</aside>
Unfortunately, CMS often fails to cover indirect and ongoing costs, including the Motor-BCI-reduceable attendance care and lost wages. Patients must resort to alternative sources of funding, such as Social Security Disability Insurance, Supplemental Security Income, and charitable organizations (ALS Association, 2017).
Beyond those with severe paralysis, WHO and UNICEF (2022) report a staggering 363 million individuals with mobility and communication difficulties. While these difficulties may result from other causes, paralysis is among the leading ones.
For instance, Armour et al. (2013), co-authored by the US Department of Health, found that 1.7% of the US population live with paralysis. Scaling the prevalence to the global population yields 136M individuals with paralysis which restricts their mobility.
“Stroke was the leading cause of paralysis, affecting 33.7% of those with paralysis, followed by SCI (27.3%), multiple sclerosis (18.6%), and cerebral palsy (8.3%).”
Further, 85% of these individuals are unemployed. Solutions that help them overcome their paralysis may lead to a monumental economic impact.
On the communication front, 97M individuals would benefit from AAC due to their struggles with expressing their thoughts verbally (Beukelman and Light, 2020).
Unsurprisingly, those most impaired by paralysis also generate the highest costs of care. In a world where economic value outconcerns individual wellbeing, the 1 million LIS individuals are 'lucky' to be considered the first recipients of Motor BCIs by all who currently develop them (Presentation Materials, Export Controls for BCI Conference at the U.S Department of Commerce, 2023).
<aside> 🔮 However, the larger 21 million segment of severely paralyzed also fails to perform most activities of daily living despite often preserving some speech and movement. Should a convincing application arise to minimize their suffering or costs, they too could find immediate benefit.
</aside>
The remaining populations will have to 'wait their turn' as Motor BCIs mature. While many are uneasy with the idea of receiving a brain implant, this opinion may be highly motivated by the perceived risk of developing complications.
It is natural to be skeptical of an early-phase technology, especially one that is expected to venture into and coexist with a body part most central to who we are. However, invasive procedures depositing an implant are not uncommon, some are even pursued for aesthetic reasons.
In 1982, the FDA placed breast implants in the rigorous Class III category (the same as the most invasive Motor BCIs) for reports of adverse events in the medical literature. 24 years later, the FDA lifted restrictions on silicone breast implants and, in 2010, they became the most popular form of plastic surgery in the US (Reuters Staff, 2012).
Now, over 500,000 people volunteer each year to undergo invasive breast surgeries for self-affirming aesthetic reasons in the US alone. What is more, these interventions are elective and therefore not covered by insurance. The mean out-of-pocket cost of breast implants is $4,294 (Aesthetic Society, 2022; American Society of Plastic Surgeons, 2022).
Tens of millions of women live with breast implants, a reality unimaginable to most folks of the 20th century. Closer to the brain, nearly a million people live with a cochlear implant, restoring one’s hearing by feeding an electrical current to the periphery of the central nervous system (NIH, 2019). In a similar manner, even inside the brain, several hundred thousand people live with an aforementioned DBS, primarily reducing the symptoms of Parkinson’s disease and essential tremor (Lozano and Lipsman, 2013).
Figure 4; Lozano and Lipsman, 2013: “Yearly Growth in the Number of DBS Publications from 1980 to 2011” Reprinted from Figure 1.