Live Text in Photos

Sadly, users of pre-2018 iPhones are going to be left out of this one, which is too bad as it’s one of the coolest new things about iOS 15.

The concept here is very straightforward — your iPhone will now find and recognize text found in any of your photos. In typical Apple fashion, however, it’s the implementation that takes this beyond a simple OCR technique into a truly magical new feature.

Firstly, for privacy reasons, all this processing is handled directly on your iPhone, using its powerful A-series CPU, so it’s no surprise that you need an A12 Bionic or better to pull it off.

However, what makes this really exciting is the fact that it works in real-time directly through your Camera app.

In other words, you can point your iPhone camera at anything that contains text, and it will recognize it, instantly.

You can then tap a button to bring up the found text and copy it right out of your Camera app, and it will land on your clipboard, ready to paste as normal text into any other app.

It can even recognize handwriting, although obviously, it’s going to be less perfect at reading everyone’s chicken-scratch.

That’s not all, though. All the iOS Data Detectors are at work here too, so if you tap on a recognizable phone number, you can tap on it to place a call directly. Ditto for things like email addresses, websites, and dates, all of which can automatically open the appropriate apps when tapped.

This naturally works for anything stored in your Photos app too, but also any other app that shows photos through the standard iOS media viewer. For example, you can pull Live Text from just about any photo you pull up in Safari or Mail just by tapping and holding on it to bring up the context menu, and then choosing “Show Text.” Sadly, however, this doesn’t work in all apps, since many like Facebook and Instagram take over the built-in iOS media controls in favour of their own.