Apple’s Exploring Sci-Fi-Inspired Virtual Touchscreens for AR Glasses

Apple may be looking into new technology that could turn any surface into a touchscreen. These virtual controls were detailed in a recent patent application.

The patent application, filed in 2016 and made public this week, describes a way of displaying a touchscreen on an object. This interface would only be visible to the person wearing an augmented reality device and not to the people around them.

Apple explains that this type of interface would be necessary as augmented reality displays get smaller and smaller. With these tiny displays, it’s almost impossible to interact with them as a touchscreen. For example, you can view content on your Apple Glasses, but it would be impractical to take off your AR glasses and start tapping on their displays.

“[However, a] current trend is that displays for AR are becoming smaller and/or they move closer to the retina of the user’s eye,” it continues. “This is for example the case for head-mounted displays and makes using touch screens difficult or even infeasible.”

Apple in its patent application

As a result, hardware manufacturers have to develop a novel method of interacting with these AR devices. Apple’s solution is to project the interface onto an object in your environment. You then could tap or touch these controls like you would on a keyboard or a touchscreen device.

In one solution, Apple would require the user to wear a touch-sensitive device on their fingers. In another solution, the glasses would use a camera to track a person’s fingers as they move. To determine whether a virtual button has been pressed, the technology could use heat to monitor temperature changes in the object that is being pushed. The longer a press, the warmer the object will become.

This patent application contains some exciting technology, but it also shows that Apple has been thinking about augmented reality for longer than we realize. This research and development is expected to culminate in a pair of smart glasses and possibly a virtual reality headset. According to recent rumors, the company could debut its augmented reality smart glasses in the next year or two.