This works just like other IM apps which do this. Skype and WhatsApp come to mind. If you have the option to send these notifications turned on, and you start typing, your iOS device sends a message to the device on the other end that you're currently typing a message, which gets displayed as "typing..." or "..." to the user.
This is possible because the Messages app on iOS works on both the phone's SMS network and Apple's proprietary iMessage IM network. The app automatically figures out whether to use the iMessage IM network or SMS, based on who you're sending the message to.
If the person is using iMessage, it defaults to that, saving both of you money. If all you have for the person in Address Book is a phone number that isn't an iPhone with iMessage set up, it uses plain old SMS. Pretty smart. BTW iMessage messages are blue, SMS are green.
Messages sent over iMessage move over the internet, rather than the cellphone's SMS network, so it'll use either your cell phone's data plan, or Wi-Fi, and maintain a relatively persistent connection to the user on the other end.
Because of this, it's trivial for your iOS device to periodically ping the device on the other end, and show the user that message, while you're entering text. This wouldn't be easy over the SMS network. AFAIK, the "..." shows up as long as you're entering text at the keyboard. If you stop typing, even if there's text already entered, it'll disappear, then reappear when you start typing again.