Control Your iPhone With Your Mind? Apple Is Working On It
Toggle Dark Mode
Yesterday, Apple gave us a sneak peek of all the new accessibility features coming in iOS 19. Nestled among handy new tools like an Accessibility Reader and Live Captions on Apple Watch was the mention of a new protocol to support brain computer interfaces (BCIs).
If you think that sounds like the stuff of sci-fi movies, you’re right. However, as a company that’s always trying to skate to where the puck is going to end up, Apple is already working on what it refers to as “an emerging technology that allows users to control their device without physical movement.”
While Apple’s accessibility announcement made only a passing mention of this protocol, The Wall Street Journal’s Rolfe Winkler was provided with a deeper look into what Apple is up to, and it turns out it’s more than just a passing fancy.
Earlier this month, Apple senior VP Eddy Cue pondered whether we’d even need an iPhone in 10 years, but it looks like the first stage of that will be letting you control your iPhone with nothing more than neural impulses — that is to say, merely by thinking.
Apple’s goal for bringing a Brain Computer Interface to the iPhone is purely about accessibility for now, but that’s because the technology is so bleeding-edge that it’s unlikely to be adopted by anyone who doesn’t have a serious need for it. As things stand now, these BCIs aren’t things you simply strap on your head — they’re devices that need to be surgically implanted in your brain.
This would follow the groundbreaking work by Elon Musk’s Neuralink, which successfully implanted a brain chip in a man with paralysis last year that allowed him to control a computer mouse using his thoughts — and play chess for the first time in years after a diving accident left him fully paralyzed below his shoulders eight years ago.
Apple hopes to bring similar capabilities to the iPhone, iPad, Mac, and Vision Pro for the tens of thousands of folks who can’t operate them using their hands.
The company is taking early steps to enable people to control their iPhones with neural signals captured by a new generation of brain implants. It could make Apple devices more accessible to tens of thousands of people who can’t use their hands because of severe spinal cord injuries or diseases such as amyotrophic lateral sclerosis, or ALS.
Rolfe Winkler, The Wall Street Journal
The iPhone and iPad are already on the leading edge of accessibility features compared to most competing devices, with built-in features like Voice Control and Eye Tracking, plus support for a wide range of assistive technologies — external devices like braille displays, hearing aids, switches and switch interfaces, and even smart canes.
With existing frameworks for these devices already in place, Apple doesn’t even have to do much to support BCIs. Ultimately, they’re just another type of switch — an assistive technology that lets users control an electronic device using external physical buttons.
In fact, Apple already lists support for the Control Bionics NeuroNode, an external wearable sensor that comes very close to a BCI, using the body’s EMG signals or 3D spatial awareness to control an iPhone, iPad, or Mac.
However, Apple naturally wants to take this to the next level, and it’s working with Synchron, a startup that makes a “stent-like” BCI that gets “implanted in a vein atop the brain’s motor cortex.” Dubbed the “Strentrode,” it can read brain signals and translate them into selecting icons on a screen, and it interfaces with the switch control feature that’s already in all of Apple’s operating systems.
The Stentrode implant has already been tested in human trials with Apple devices, Winkler reports:
Mark Jackson, an early tester of the Stentrode implant, was able to peer over the ledge of a mountain in the Swiss Alps and feel his legs shake. Jackson can’t stand up, and he wasn’t in Switzerland. He was wearing an Apple virtual-reality headset, which was connected to his implant.
Jackson can’t travel from his home outside Pittsburgh because he has ALS. Still, he is learning how to control his iPhone, iPad and Vision Pro headset thanks to a connection between his Stentrode implant and Apple’s various operating systems.Rolfe Winkler, The Wall Street Journal
Synchon has several other videos on its YouTube channel, including one that shows a man named Rodney, who is living with ALS, using its BCI along with an Apple Vision Pro and NVIDIA Holoscan to feed his dog.
Nevertheless, while this certainly feels like the future, Apple has a long road ahead of it before these will become routinely available to those with severe mobility disabilities. Jackson said the technology is still very preliminary and lacks the ability to mimic moving a cursor or using touchscreen gestures, making navigation much slower and more cumbersome.
This is where Apple’s new protocol comes into play. According to Synchron Chief Executive Tom Oxley, Apple is developing a new standard designed explicitly for BCIs, which it plans to release later this year. That protocol will advance the state of the art from merely emulating a mouse to providing a more complete control interface.
While Musk’s Neuralink only performed its first human trial early last year, Sychron has implanted Strentrode in 10 people since 2019. However, Neuralink’s N1 has proven to be faster as it’s substantially more advanced, with 1,000 electrodes to pick up neural activity, compared to only 16 in the Strentrode. However, the greater number of electrodes makes the N1 considerably more invasive — they need to be implanted deeper inside the brain.
Further, while Apple and Synchron are focused on improving the lives of people with severe mobility disabilities, Musk’s vision for Neuralink is far more ambitious, with plans to make his implants available to all people so they can become the equal of superintelligent AI systems.
Such devices will need to be approved by the US Food and Drug Administration (FDA) and other health regulators around the world before they become available even for medical purposes. Analysts such as Morgan Stanley expect that we’ll see the first commercial devices available by 2030, although Synchron’s Oxley is optimistic that the Strentrode will receive approval even sooner.
