AppleScript - How to identify buttons and press them?

Question

I'm trying to automate the running of techtools via AppleScript. I have got it working but only with full keyboard access (using tab and space to navigate the UI) but not all machines have this enabled when I first turn them on so I need a way of activating the run check computer.
I assume this is possible with the "click button" function but I cannot work out how to find out which button is which with accessibility inspector. Here is the layout of TechTools and you can see in the bottom left is the button I want to press to start the tests.

And here is the inspector info on the button.

As I said, I can activate this fine when full keyboard access is enabled because it's just a case of copying keyboard strokes.

I'm not sure if i'm misunderstanding the concept of identifying the buttons within the program or how to specify which window it's in (I assume it's window 1.)

This is an example of the type of script I've been writing to attempt to achieve this (I know this one just minimises the window.) Do I keep increasing the value of the button until it eventually just runs check computer? When I tried button 4 it activated an option under File on the top bar.
Any help would be greatly appreciated.

tell application "System Events"
    tell process "TechTool Pro 9"
        click button 2 of window 1
    end tell
end tell

Answer 1

Here's a little trick:

Obtain the screen coordinates of that button. You can do this by initiating a screen capture with crosshairs, usually by pressing ⌘⇧4 (⟨Cmd⟩⟨Shift⟩⟨4⟩). Navigate the cross hairs over the button within its clickable region.

I did a similar thing with a dialog box in Script Editor:

You can just about make out the coordinates of my button are {896,674}.

Next, in Script Editor, enter this line of code, inserting your coordinates as appropriate:

    tell application "System Events" to click at {896, 674}

I can assume you already have the right accessibility privileges granted from the context of your post. Therefore, you can simply run this script.

Just ensure the button is visible before you run the script, and that it hasn't changed position on the screen. Make sure there are no windows obscuring or overlying it.

All being well, two things will happen:

1. The system will issue a click and it'll click that button;
2. In the results pane at the bottom of the Script Editor window, you'll see something like this:

    button "Done" of window "Open" of application process "Script Editor" of application "System Events"

That is the reference to the button object, which you can copy and paste into your script.

It works for any GUI item on the screen: just isolate its coordinates, issue a click, and it will return the object reference.

UPDATE:

Having just acquired a copy of TechTool Pro version 8, I have learned that the application is not GUI-scriptable. Therefore, this explains why the trick I described failed to return an object reference (because there is none), and unfortunately, you won't be able to automate the running of TechTools using AppleScript.

Answer 2

Sikuli

While this is possible with AppleScript, consider using Sikuli for this task:

Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code.