Popular third-party apps like Snapchat and Pokemon Go rely on it. Facebook and Google have invested heavily in platforms for it. Now, it looks like Apple has lived up to its word and officially entered into the augmented reality fray.
What Is ARKit?
During this year’s WWDC keynote, Apple announced a developer toolset called ARKit, a platform that will bring advanced augmented reality capabilities and apps to iOS devices. Apple vice president Craig Federighi boasted that, when launched with iOS 11, ARKit will be the “largest AR platform in the world” by virtue of its availability across the hundreds of millions of phones and tablets comprising the iOS ecosystem. It also means that with the iOS 11 update, your iPhone and iPad will harness more computing power behind their cameras to perform tons of new tricks. For instance, ARKit enables “fast and stable motion tracking” that allows virtual objects to look like they’ve actually been placed in reality.
In an onstage demonstration of ARKit’s spatial recognition capabilities, Apple showed an iPhone being used to cast a cup of coffee onto the surface of a real-world table with realistic perspective. A virtual lamp was then placed on the table, and when it was turned on, the cup’s shadow reacted and changed within the environment accordingly.
How ARKit Works
ARKit leverages the cameras, motion sensors CPUs, and GPUs already built into iOS devices. For instance, ARKit fuses data from the camera and motion sensors to accurately detect the world around a device and sense how it is moving in relation to its surroundings (which is called visual inertial odometry– or VIO). In addition, ARKit brings deep software support for plane estimation with basic boundaries, ambient lighting estimation, scale estimation, and Xcode app templates. This allows ARKit-powered iPhones and iPads to detect horizontal planes like tables and floors, and place objects on them. ARKit also uses the camera sensor to measure the ambient lighting and apply the right amount of light to virtual objects.
The aforementioned features will let developers use ARKit “to build detailed and compelling virtual content on top of real-world scenes for interactive gaming, immersive shopping experiences, industrial design and more”, according to the press release. So far ARKit is compatible with Metal, Unity, and third-party tools like Unreal Engine, and Scenekit.
Apple CEO Tim Cook expressed enthusiasm for augmented reality last year following the release of the hugely popular Pokemon Go app. Then, he noted that AR “will be huge” and promised that Apple would ramp up its involvement in the space: “We have been and continue to invest a lot in this. We are high on AR for the long run, we think there’s great things for customers and a great commercial opportunity. The number one thing is to make sure our products work well with other developers’ kind of products like Pokemon, that’s why you see so many iPhones in the wild chasing Pokemons.”
As promised, Apple noted today during its ARKit introduction that Pokemon Go will eventually work with it, meaning the beloved game is in for an exciting upgrade that will bring depth and accuracy to gameplay. The company also demoed a game made using Unreal Engine 4 and developed by Peter Jackson’s Wingnut AR studio, which is due for release later this year. On top of that, Apple mentioned that IKEA and Lego have already begun developing apps on ARKit. The former will likely be a shopping app while the latter will be a Batman Lego app.
We’ll most likely learn much more about Apple’s plans for AR when it unveils its tenth anniversary iPhone later this year. Developers can get started with ARKit earlier by downloading the betas for iOS 11 and Xcode 9.