Furry Friends, Laundry Care, and More | Your iPhone’s Image Recognition Will Get Much Smarter in iOS 17

Of all the exciting new things coming to iOS 17, perhaps the most underrated are the machine learning improvements that promise to let your iPhone’s camera identify entirely new categories of information.

As usual, Apple doesn’t have the time to cover everything in its WWDC Keynote, and that was even truer this year as it set aside the lion’s share of the presentation to show off its groundbreaking Vision Pro mixed-realty headset.

However, some of the things that Apple barely mentioned — or didn’t mention at all — seem made for the Vision Pro as much as the iPhone. This includes new Visual Look Up capabilities that can detect many more types of information in your photos — and now videos, too — as well as “facial” recognition for your furry friends.

Pets Are People, Too

One of the new iOS 17 recognition features Apple mentioned in passing during the WWDC keynote is that you’ll soon be able to add cats and dogs to your photo library alongside the other important people in your life.

To this end, the “People” album in iOS 17 is now called “People & Pets.” If you open it and scroll down to the bottom, it now includes the cats and dogs found in your photo library.

Adding names and photos for your pets works just like it does for other faces. You can tap on your pet, enter its name, find additional matching photos, and mark it as a favorite.

While Apple says this feature is designed primarily to identify cats and dogs, I had no problem getting it to pick up a Holland Lop rabbit that was once a member of my family (RIP, Napsters), effortlessly gathering every matching photo of bunny across the years and placing them in a single album.

Of course, rabbits are furry creatures that would visually fit into the same general category as cats and dogs. I had no success getting it to pick up more exotic animals like birds, lizards, and turtles.

Expanded Visual Look Up

Apple introduced Visual Look Up in iOS 15 as a US-only feature, later expanding it internationally in iOS 15.4. With it, iPhone users can open pictures of animals, plants, flowers, or landmarks and tap to get more information about what they’re looking at.

Visual Look Up also works with albums and artists from the cover art, which isn’t surprising, considering Apple has a massive library of album artwork at its fingertips in Apple Music. Strangely, though, it doesn’t talk about this feature nearly as much as the others.

With iOS 17, you’ll also be able to perform a Visual Look Up in videos by pausing on any frame and tapping and holding on the subject you want to look up.

However, what’s even cooler is that Apple has quietly expanded the types of information that your iPhone can look up. The iOS 17 Preview page notes the ability to look up recipes for similar dishes from a picture of some food, but it turns out that may just be the tip of the iceberg.

Folks playing with the iOS 17 betas have since discovered that the Photos app will also look up other things that Apple didn’t mention, including automotive symbols on car dashboards and laundry care symbols on clothing tags.

This works just like it does for animals, plants, and landmarks. Snap a picture of your dashboard or a label on a piece of clothing, and you can open that photo and see the Visual Look Up information by tapping the info (“I”) button and then choosing Look Up Laundry Care or Look Up Auto Symbol.

This will show a list of what each symbol means. Tapping on one of those entries will take you to an appropriate page on iso.org, the International Organization for Standardization website, for a more detailed explanation of the symbol.

As you can see from the photos above, there may still be some kinks to work out; my iPhone identified something on a Moto G Stylus 5G camera bump as a “Heated steering wheel” indicator or button.

Apple hasn’t spoken about these extra Look Up features yet, so there’s no guarantee this capability will still be there by the time iOS 17 is released in the fall, but it also seems to work well enough in the current beta that there’s no reason to believe Apple would remove this capability.

The only catch is that, for now at least, the new lookup capabilities for food, auto symbols, and laundry care symbols are only available if your iPhone’s region is set to the United States. It remains to be seen when (or if) Apple plans to expand it internationally.