I found a simple solution which DOES involve a SIM card to this issue, but without the recurring fee of a SIM card. I purchased an EE SIM card for 99p at a local shop and stuffed it into the phone. On that SIM card I did get some free texts, minutes and data, but I used all the data up quickly and the minutes and texts had their free period expire. Now, this SIM was a pay-as-you-go SIM, and was not linked in any way to any of my payment details, meaning I can use it in data mode without any further cost. Simply, I turned on Mobile Data even though I had no data left and had not topped the card up with enough money to buy a data pack, and this allowed me to turn on Personal Hotspot. If I connect to this phone with another iDevice, I get captive.apple.com sign-in pop up, which can be disabled by tapping cancel, pressing the blue i next to the hostspot network under WiFi and switching off Auto-Login. Connecting again will make the hotspot stay connected and any attempt to connect to the internet from a tethered iDevice will result in being redirected to the Buy More Data popup.
Connecting via USB to a Windows PC also resulted in this popup, but simply closing it fixed it.
I can now find the IP address of the currently tethered device, for example 188.8.131.52 and work out that the hotspot's IP address is 184.108.40.206 which, connecting in an SSH app brings me to the OpenSSH command prompt! Hurray!
This was with an EE SIM, and if you are in America or elsewhere I believe EE isn't an option for you, but just because it worked with an EE SIM doesn't mean it won't also work with another kind of pay-as-you-go pick up at the store SIM.