New Patent Shows How Apple’s AR Glasses Could Handle Audio

We’ve been hearing reports for over two years now that Apple is working on an actual augmented reality device, and while there’s been a lot of back-and-forth about whether it’s going to be glasses, a headset, or both, it’s clear that the company has something groundbreaking in store.

Up until now, however, most reports have focused on the visual aspects of such a device — display screens, LED and holographic projectors, and the visor and glass design itself. However, if Apple is working on a full headset, there’s another important factor that’s easily missed, and that’s building an audio experience that can match the visuals.

Of course Apple hasn’t missed this aspect of the solution, and a new patent discovered by Patently Apple describes an “augmented reality” headset that would be able to virtually position people in a room, even when they’re not physically present, so you would hear from which part of the room they’re speaking.

The actual patent describes the feature more as a business tool to be used in conference calling scenarios in order to provide a sense of space and presence for everybody participating, but of course it’s also a natural fit for virtual reality and even augmented reality applications, especially if the rumours of Apple’s work on a gaming headset are true.

The basic idea behind the patent as written, however, would be to help users identify who is speaking during a large meeting, even if you don’t readily recognize the person’s voice — something that can often be a challenge on a busy conference call.

Multiparty audio conference calls typically include audio signals from individual callers in different locations, such that it may be difficult to recognize who is speaking by voice alone. Often, a listener uses a stereo headset to listen to the conversation, and the audio signals may be processed to enhance recognition of the different participants.

In addition to placing users spatially within a virtual meeting room, Apple’s AR headphones would be intelligent enough to recognize your head position and adjust the audio accordingly, adjusting the virtual position of other people so that they would remain in the same location relative to where you’re sitting, regardless of the position of your head.

Obviously this could also be applied in virtual reality gaming to great effect, allowing you to hear movements around you in a first-person shooter, enhancing the gaming environment with realistic spatial awareness.

Apple’s AR Glasses are Coming

Although early reports had suggested that Apple could have had an AR device out by late this year or early next, a recent “all-hands” meeting within Apple suggests that those plans have been pushed back, with the company now expected to release a full AR Headset with 3D scanning by 2022, followed by AR Glasses the year after. However, reports suggest that we could be hearing about the project as early as 2021 when Apple invites developers to participate.

It’s less clear how AR audio would fit into a pure AR glasses experience, which wouldn’t necessarily include any kind of earphone or headphone component, but since this appears to be largely a software technology, it wouldn’t surprise us to see Apple include it in a future set of AirPods that could complement the AR glasses.

Of course, this is only a patent at this point, so the usual disclaimers apply. Apple has a great many patents that have never seen the light of day; sometimes they turn out not to be workable solutions, and sometimes Apple just patents things defensively to prevent other companies from scooping up ideas first. In this case, however, we know that an Apple AR headset is all but guaranteed to arrive in the next few years, and proper virtual audio seems like an important component of a full and immersive augmented reality solution.