Digitize Text from Images with Live Text

One of the most anticipated features of Apple’s latest crop of operating systems—iOS 15, iPadOS 15, and macOS 12 Monterey—may be Live Text. You’re probably familiar with the concept of optical character recognition (OCR), which turns all the text on a scanned page into normal text, which you can then select, copy, and edit the text itself. The process was busy and required scanners. Live Text does this and more! It can scan text in Photos, Safari, and the Camera app, plus lets you search for text in photos. And it does it in not only in English, but also Chinese, French, German, Italian, Portuguese, and Spanish. I can only assume Apple will add more languages in the future.

This can happen thanks to the incredible processing power of Apple’s recent devices. Why is is called “live text”? The digitization of graphics to text takes place instantly—hence “live” text. You do need to have a later device that is running the latest OS. Apple says it works on any Mac that can run Monterey, but on the iPhone and iPad, it requires a model with an A12 Bionic processor or later, which means anything in this list (check your device in Settings > General > About > Model Name):

• iPhone XR, XS, XS Max, and later, including the iPhone SE (2nd generation)
• iPad Pro 12.9-inch (3rd generation) and later
• iPad Pro 11-inch
• iPad Air (3rd generation) and later
• iPad (8th generation) and later
• iPad mini (5th generation) and later

Live Text in Photos

I believe the main place you’ll use Live Text is in Photos. In iOS and iPadOS, when viewing a picture containing text, just touch and hold a bit of text until it highlights in blue. If the image has a lot of text, Photos displays a Detect Text button in the lower-right corner. You can tap this button to highlight recognized text. When you long-press on the text in the image, the familiar text popover then appears. Drag the blue selection handles to adjust the selection as desired or tap Select All. Then you’ll most likely want to copy the text, switch to another app, and paste. You can also do a Web search by tapping  Look Up or have the device read the text out loud by tapping Speak.

A few quick tips: Want to copy text from an app that doesn’t even let you select text? 

• Take a screenshot of the app and use Live Text to copy it. 
• If you have an animated Live Photo that contains text, touch and hold the text to select it; touch and hold away from the text to animate the photo.
• In Monterey, move the pointer over the text so it turns to a text selection cursor, then select the text, after which you can copy it as you would any other text. You can also Control-click the selected text to access Look Up or Translate.

As a bonus in iOS 15 and iPadOS 15, text from Photos that Live Text has recognized is indexed by the system-wide search. So when you pull down from the middle of the Home screen and search, a Text Found in Photos section shows images that contain the search text. Tap one to view it, and note that there’s a Detect Text button in the preview that identifies text you can select right there. Or tap the Photos button in the lower-right corner to open the image in Photos.

Live Text in the Camera App

Sometimes you may want to capture some text in the real world without having a photo to store and later delete—getting an appliance’s model or serial number is a perfect example. When Live Text detects text in the Camera app’s viewfinder, it puts a faint yellow outline around it and displays the Detect Text button. Tap the Detect Text button to freeze and zoom the selection—it even rotates sideways text to be readable—and then tap the text to select and copy it. Tap the Detect Text button again when you’re  done.

Scan Text

There’s another way to use Live Text in conjunction with the iPhone or iPad camera that’s useful if you need to insert a large amount of text directly into a document. In an app that supports Live Text, like Notes, tap in the text entry area and then tap the Scan Text button. A viewfinder appears on the bottom half of the screen, with a yellow outline around detected text, and the recognized text above. If that’s what you want, tap the Insert button to put the text into your note or document. 

Live Text in Safari and Other Apps

Because Apple integrated Live Text into its operating systems, it’s available nearly anywhere you might see an image that contains text. For instance, in Safari, you’re likely to run across such images, or you might have embedded photos in Notes that contain text. Selecting text in an image works exactly the same as in Photos—either touch and hold on an iPhone or iPad, or just select the text on the Mac. Speaking of the Mac, Live Text even works when you select an image in the Finder and press the Space  bar to view it in Quick Look.

Other Live Text Tricks

• Identify types of data: When Live Text recognizes text that contains an email address, phone number, postal address, or date, you can touch and hold it to bring up special options to send an email, call the number, map the address, or make an event or reminder.
• Translate Live Text: The Translate app in iOS 15 and iPadOS 15 is now a system-wide service, so it can translate any text you can select, including text recognized by Live Text in a photo or in the Camera app. Just select it and tap Translate.
• Hear Live Text: If you rely on VoiceOver to help you navigate your iPhone or iPad, note that you can leverage Live Text to get VoiceOver to read text in your photos out loud. Go to Settings > Accessibility > VoiceOver > VoiceOver Recognition and turn on Text Recognition. Then, while viewing a photo, tap the text in the photo to have VoiceOver read to you.

It sounds like a lot and it may take a little while to remember that you can select and copy, look up, or translate nearly any text you see in a picture or in your camera viewfinder, but once you’ve tried Live Text a few times, I think you’ll find it surprisingly useful. You just have to remember you can select text from an image. I am still training myself on this concept. The more I use it, the more useful I find Live Text. Apple Provides Temporary iCloud Backup Space for New Device Transfers.