Thus, my question is whether there’s a way for me to get "at most 5 results" in a single query?
No, it cannot be done in a single query.
If you want to loop through a given count downwards then the following example AppleScript code is how I'd do it:
set maxNumber to 5
set searchString to "a"
set matches to missing value
repeat with i from maxNumber to 1 by -1
if matches is equal to missing value then
try
tell application "Notes"
set matches to id of ¬
(notes 1 thru i in default account ¬
whose name contains searchString)
end tell
end try
if matches is not equal to missing value then exit repeat
end if
end repeat
return matches