I've got an iPhone 4s from a provider, but they gave it to me without a sim card. It worked but I am wondering if I could buy a sim card, put it in the slot and use the phone with a different carrier, like a prepaid one?
If you got your iPhone from a carrier, it is almost certainly locked to that carrier, so you can only use it with a SIM from that carrier. The carriers subsidize the cost of the phones, for the opportunity to make money selling you the data/phone plan. This is why purchasing an iPhone from a carrier typically costs $0-$200 while an unlocked phone costs $600-$1000.
It can be possible to unlock iPhones. An unlocked phone can be used with a SIM from any mobile carrier, including carriers from other countries.
Unlocking is completely independent of jail-breaking, which relates to modifying the software to accept non-Apple-approved apps.
Usually, if you want to unlock your iPhone and use it with any carrier/provider you want, there are some conditions under which you can officially obtain the unlock. As far as I know, you are eligle for an unlock if the contract with your original carrier has expired over 24 months (some carrier 30), and , of course, if you are the legal owner of the contract. Then, you can contact your carrier and ask for the unlock. They can eventually ask for additional fees, others don't.
Be aware of the bunch of sites that promise an official unlock under payment. Most of them won't work.
You can find terms and conditions of the unlock of your iPhone on your provider site.