ARKit is Apple’s framework for integrating augmented reality experiences into an iOS app or game. ARKit uses the device’s cameras and overlays 2D or 3D elements into the real-world scene that the camera is capturing.
Apple recently updated ARKit to version 3.5, adding three significant features that take advantage of the 2020 iPad Pro and its new LiDAR scanner. It’s a game-changer for augmented reality on the iOS platform and likely will roll out to devices beyond the 2020 iPad Pro. Continue reading to learn what ARKit 3.5 is, and why it matters.
What Is the LiDAR Scanner?
LiDAR is short for Light Detection and Ranging. As its name implies, this scanner can calculate the distance of nearby objects by measuring how fast light reaches the object and is reflected back to the sensor. This sensor is new to the iPad but is not a new technology. It’s quite common on drones where it’s used to map terrain. On the iPad, LiDAR is used for a whole lot more than just terrain. It opens the door for amazing augmented reality experiences. More on that below.
Scene Geometry API
One of the more significant new features in ARKit 3.5 is the scene geometry API. The API uses the iPad’s LiDAR scanner to generate a detailed 3D map of a person’s environment with the ability to distinguish between objects like walls, doors, windows, and more. Being able to distinguish objects with such granularity allows developers to not only overlay their 2D or 3D objects but blend them into the scene.
The Instant AR in ARKit 3.5 that leverages the fast plane detection feature of the LiDAR Scanner to add 2D or 3D objects to a scene in an instant. Users won’t have to wait for a comprehensive scan of the surroundings before 2D or 3D objects appear. They will show up in the real-world view almost immediately. This feature is enabled automatically in all augmented reality apps. No code changes are required.
Improved Motion Capture and People Occlusion
ARKit 3.5, along with the LiDAR scanner on the iPad Pro, is better at estimating the depth of people in the surroundings and the height of objects in motion capture. This feature also is enabled without any code changes.
What’s in It for iPhone Users?
ARKit 3.5 may focus on the new 2020 iPad Pro and its LiDAR scanner, but that’s only a small portion of the framework. There are features that iPhone users can enjoy, such as the simultaneous use of the front and back camera in augmented reality experiences. ARKit also can track up to three faces at once in a scene and can follow a person’s motion in real-time. It even also allows developers to create a green screen effect by allowing augmented reality objects to pass behind and in front of people in the scene.
The new LiDAR scanner and ARKit expansion is only the beginning of Apple’s foray into augmented reality. With WWDC 2020 right around the corner, we expect to hear more about augmented reality and how it will be expanded into iOS 14. AR may be the hot new feature we see in future iOS apps and games.
There’s also new iPhone hardware expected later this year that may include time-of-flight 3D camera technology. If these augmented reality camera modules do roll out to the iPhone and other iPad models, we expect cutting edge mobile AR technology to become as conventional as the Ultra-Wide lens.