I'm going to assume there's no pattern to when this happens and it's totally random.
My first thought would be that it's a connection issue (remembering that your watch is designed to communicate with your iPhone). So I'd go through the process of unpairing and then restoring your Apple Watch to see if that resolves the issue.
IMPORTANT: - Read through all of this answer before commencing the process so that you know what's involved.
You can un-pair your Apple Watch from the Watch itself or from your iPhone. Since you're having frequent unresponsiveness from your Watch, the following steps are for doing this from your iPhone.
Un-pair and restore your Apple Watch
- On your iPhone go to the Apple Watch app
- Make sure you're on the My Watch screen
- Tap on your watch name at the top of the screen
- Tap on the info icon (i.e. the amber circle with an 'i in it)
- Now select the red Un-pair Apple Watch option
- You'll be prompted to confirm the un-pair process
- If you have an e-sim activated with a mobile service, choose to keep your plan
- Disable Activation Lock by entering your Apple ID password and tapping Un-pair
- Your watch data will begin backing up to your iPhone
When complete, your watch will:
- be unpaired from your iPhone
- be unpaired from any Bluetooth devices
- no longer have any Apple Pay cards attached
- have no passcode
- be reset to Apple's factory settings
OPTIONAL: You may want to backup your iPhone as it's now the only place all of your Apple Watch data is stored
- Switch on your Apple Watch
- Put it on
- Now make sure your iPhone is close to your Apple Watch
- Once the "Use your iPhone to set up this Apple Watch" prompt appears, tap on Continue. (NOTE: If the prompt doesn't appear, open the Apple Watch app on your iPhone and tap the Start Pairing option)
- Follow the prompts to pair your Watch to your iPhone and to set it up
Now start using your watch again to see if it works properly. If it doesn't I'd call Apple Support. They can actually run diagnostics on your Apple Watch remotely (you will need to have your iPhone with you as well in order for them to do this). It doesn't take long to do and will help identify if there are any hardware issues.