Zoom and Live Recognition on Vision Pro

As you’re probably aware by now, Apple takes privacy very seriously. The Vision Pro created a whole new set of privacy concerns — imagine having always-on cameras scanning your entire room and tracking where your eyeballs are looking — so Apple made sure that none of this data would be available to third-party apps.

Unfortunately, protecting user privacy often runs contrary to usability, and Apple’s limitations also ruled out apps being able to use the cameras to benefit users with accessibility needs. While Apple still isn’t about to open the floodgates to let developers get unfettered access to the Vision Pro cameras, it’s created a new API in visionOS 3 that will let apps access the main camera to provide live, person-to-person assistance for apps like Be My Eyes, helping users who are blind or have low vision better understand their surroundings hands-free.

Apple is also updating Zoom to let users magnify everything in their field of view and adding Live Recognition in visionOS to provide VoiceOver descriptions. This will let visionOS describe their surroundings, help them find objects in the real world, read documents, and more.