Apple Working on a New ‘Gaze Detection’ Feature to Protect Your iPad from Curious Onlookers

As amazing and revolutionary as the iPad is, the ability to easily use it anywhere actually presents a slightly unique privacy problem, since its large screen makes it easy for others to spy on whatever you’re reading or working on.

Of course, many iPad users many not care, since reading news and casual browsing isn’t exactly confidential, and your mom’s banana nut loaf recipe probably isn’t the big family secret you’ve been led to believe it is.

On the other hand, however, the iPad is just as much a business tool, and there are many folks who may legitimately be using to read and compose confidential emails, spreadsheets, and presentations. Besides, even your casual web surfing is really nobody else’s business.

While some users have turned to special screen protectors that improve privacy by making it more difficult to read the iPad unless you’re looking at it head-on, these aren’t entirely foolproof, and it’s hard to find one that doesn’t also negatively impact your use of the iPad, either by cutting down the brightness, distorting colours, or making it more difficult to use the Apple Pencil.

Fortunately, it looks like Apple has a better idea up its sleeve. A recently granted patent reveals that the company has been working on a way of obscuring the information on the iPad display to show only the portion that the user is actually looking at.

The patent, 10,996,748, was unearthed by AppleInsider, and is titled “Gaze-dependent display encryption” As the name suggests, this would introduce the ability to track where the user is looking to determine which parts of the screen to display in readable form, while leaving the rest disguised in some way.

Each display frame that is displayed on the electronic device display may include a clear-display region around the user’s gaze location and an obscured region outside the clear-display region. In this way, only the display content that the user is actively viewing is recognizable and understandable and an onlooker such as an unwanted observer looking over the user’s shoulder is unable to understand what is displayed.

Apple patent filing

Although the patent doesn’t go into the specifics of the technology that would be used, it’s a pretty safe bet that this would rely largely on the TrueDepth camera technology that’s been used in the iPad Pro since 2018.

Modern iPhones and iPads are already capable of using the front TrueDepth camera to read and track eyeball movements well enough to know if you’re actually looking at your iPhone or iPad when unlocking it, as well as disabling automatic locking when it determines you’re actually looking at the screen. iOS 14 also added a new Eye Contact feature to FaceTime, using artificial intelligence to adjust your gaze so that it looks like you’re establishing natural eye contact with the other party.

So with these cool innovations already in place, it seems like the TrueDepth camera technology is already up to the task of handling the “gaze detection” aspect of this feature. The bigger trick is the “display encryption” side of things, which would need to be quick enough to allow for a natural reading experience as the users’ eyes moved around the virtual page.

However, with all the power packed into the M1 chip that’s now found in the latest iPad Pro models, it’s hard to believe that it couldn’t keep up. However, there are likely still software-level things like machine learning algorithms that need to be considered to bring it to the level of polish that Apple would almost certainly require.

In fact, Apple wouldn’t even be the first to use gaze detection in this way. Samsung famously debuted a “Smart Scroll” feature on the Galaxy S4 over eight years ago that could automatically scroll your view as you moved your head up and down. Unfortunately, however, like most of the features that come as a result of Samsung’s “everything-but-the-kitchen-sink” approach to adding features, it worked just well enough to be frustrating when it didn’t — which was most of the time.

Since Apple takes time to do things right, it wasn’t until the TrueDepth camera took its place on the iPhone X that Apple even began considering gaze detection type features, and it’s still not trying anything quite as ambitious or gimmicky as Samsung’s smart scrolling.

Instead, Apple is looking at gaze detection features in a variety of other practical ways, with other recent patents suggesting it could allow notification to be automatically marked as read, without any other user interaction. This would be similar to the way that modern iOS devices already unmask notification previews once the user is recognized via Face ID — a very cool feature that many of us now take for granted.

In this case, the patent describes a few different ways in which the other portions for the display could be obscured, ranging from simply scrambling the text to make it unintelligible to blurring or adding other graphic filters over the unviewed areas. Whatever the method, however, the goal would be to disguise those portions of the screen, rather than blacking it out entirely, such that an onlooker’s attention wouldn’t be automatically drawn to the readable area.

The patent lists its inventors as Mehmet Agaoglu, Cheng Chen, Harsha Shirahatti, Zhibing Ge, Shih-Chyuan Fan Jiang, Nischay Goel, Jiaying Wu, and William Sprague, and was filed in September 2019, and first appeared as a patent application in March 2020.

Of course, the usual caveats for all Apple patents also apply here. Apple has patented lots of ideas over the years, and continues to do so, and the existence of a patent is never a guarantee that the technology will make it into production. However, some ideas seem more plausible than others, and with Apple’s strong focus on user privacy, this is certainly something we could see happening if the company can work out all the kinks.

[The information provided in this article has NOT been confirmed by Apple and may be speculation. Provided details may not be factual. Take all rumors, tech or otherwise, with a grain of salt.]