try this little syntax change:

    on setfmFileOptions(app_name)
    
        -- choose the Developer Utilities menu item from the Tools menu of FileMaker Pro Advanced
    
        do_menu(app_name, "File", "File Options...")
    
    
        -- click the "Log in using" checkbox:
        tell application "System Events"
            tell process app_name
                repeat while (exists checkbox "Log in using:" of tab group 1 of window 1) is false
                    delay 1
                end repeat
                click checkbox "Log in using:" of tab group 1 of window 1
            end tell
        end tell
    end setfmFileOptions

You can't use window 1 without telling what process window 1 is in. You'll have to specify app_name as a parameter now.

So, like this:

    setfmFileOptions("FileMaker Pro Advanced")