21

Recently I've found on my Mac that I can easily highlight text in an image that wasn't possible before.

Is there also a built-in CLI option or an AppleScript option to address the OCR program with which I can achieve the same results as tesseract?

2

3 Answers 3

16

Shortcuts.app has an "Extract Text from Image" action.

enter image description here

There's also a shell command shortcuts to run a Shortcut.

8
  • Shortcuts seems to only be for iOS though not macOS, right?
    – mowwwalker
    Sep 20, 2022 at 21:43
  • 2
    @mowwwalker No, Shortcuts.app is in macOS, as of Monterey.
    – benwiggy
    Sep 21, 2022 at 6:55
  • 2
    @TryTryAgain Not in the Gallery -- Create a new Shortcut, and add it from the list of actions.
    – benwiggy
    Nov 10, 2023 at 17:35
  • 1
    @TryTryAgain Probably best to read up on Apple's help pages for how Shortcuts work. You have to construct a workflow, with an input, an action and an output.
    – benwiggy
    Nov 10, 2023 at 19:14
  • 3
    This seems promising ;) youtube.com/watch?v=BZVlifUpr_c Nov 10, 2023 at 19:24
3

This is core of a simple cli tool to extract text from an image:

 let img = NSImage(byReferencing: url)
    guard let imgRef = img.cgImage(forProposedRect: nil, context: nil, hints: nil) else {
        fputs("Error: failed to convert NSImage to CGImage for '\(url)'\n", stderr)
        return 1
    }

    let request = VNRecognizeTextRequest { (request, error) in
        let observations = request.results as? [VNRecognizedTextObservation] ?? []
        let obs : [String] = observations.map { $0.topCandidates(1).first?.string ?? ""}
        try? obs.joined(separator: "\n").write(to: url.appendingPathExtension("md"), atomically: true, encoding: String.Encoding.utf8)

        fputs("got page obs is \(obs)", stderr)
    }
    request.recognitionLevel = VNRequestTextRecognitionLevel.accurate // or .fast
    request.usesLanguageCorrection = true
    request.revision = VNRecognizeTextRequestRevision2
    request.recognitionLanguages = ["de"]
    request.customWords = ["more", "Worte", "Wort"]

    try? VNImageRequestHandler(cgImage: imgRef, options: [:]).perform([request])

from this full working project

0
3

Here are the steps for a shortcut that lets you drag your mouse over an image and receive the text on your clipboard -- basically, copy and paste text from anything on your screen, including other people's screen shares!

Creating the shortcut

  • Open the Shortcuts app and start a new shortcut.
  • Add Action: Take Interactive Screenshot, (Custom)
  • Add Action: Extract Text from (Screenshot)
  • Add Action: Copy (Text from image) to Clipboard
  • Add a title by clicking in the title bar.

Adding your shortcut to the Dock

Go back to the main window, right click the shortcut, Add to Dock.

(Optional) Adding a beep when the clipboard is ready

  • Find a sound in /System/Library/Sounds and drag it to the top. I used /System/Library/Sounds/Basso.aiff.
  • Add Action: Play Sound. Select variable, click "File". (There will not be a line connecting this action to the others -- that's OK.)

(Optional) Assigning a keyboard shortcut

Open Details (click i on the right hand side). Note that the shortcut takes a second to run after its icon appears in the top menu bar.

I have found this shortcut sometimes stops working, in which case I have had to close Shortcuts completely, find the keyboard shortcut in System Preferences > Keyboard > Keyboard Shortcuts > Services > Shortcuts, then click into it and reassign it.

Screenshot of your new shortcut

image of ocr shortcut

You must log in to answer this question.