I am trying to use an old iPhone 3GS as an iPod touch. I left in the old sim card and connected it to iTunes but it will not activate.
This should work, but might not be ideal - get a pay-as-you-go (i.e., non-contract) SIM with a small amount of credit on it (I suppose not everyone has them lying around) and activate it with that (it needs to be on the same network that the phone is locked to) and then disable all the 'phone' functions. It should then function just fine.
Per Apple's support documentation you'll need to activate the device using a SIM from another activated iPhone to activate the device. If you don't have access to another SIM card you could consider taking the device to an Apple store, or a mobile vendor that carries iPhones. If you don't have access to either of those, as a last resort you could download Redsn0w and activate the device without a SIM. The last method is not supported by Apple, but will accomplish your task.
If you remove the cellular service attached to a Sim card in an iPhone, it will continue to work Foley with only one small exception:
phone calls, SMS and cellular data will clearly be off so you can't use any of those functions
You will want to leave the Sim card inside since it is part of the self activation if you restore the iOS. At that point the phone will receive Bluetooth and Wi-Fi and work like it was in airplane mode and you turned those two radios on solely.
The cellular radio will still be active for emergency 911 calls, and you will want to turn off the radios in areas of low coverage because the device will be using a lot of energy to transmit and keep in touch with distance out hours just as if you had maintained service.