The behavior you describe is inconsistent. In my experience, sometimes the podcast will continue and sometimes the iOS device will switch to the music app. I believe (but haven't confirmed) that this depends on whether or not the podcast app, be it Apple's or third party, is still considered active by the system. When it occurs I just open my podcast app and playback of the the last podcast resumes (unless I was streaming it in which case I may have to hit play.)
Because this is a memory management issue there is no way to force the behavior you seek using a dedicated podcast app. However, if you uninstall Apple's Podcasts you can continue to download podcasts using iTunes and can move them over to your iOS device with a playlist. This solution is less than ideal—for example you can't download fresh podcasts directly to the device.
iOS 7 UPDATE: It is my understanding that under iOS 7 you no longer have the ability to listen to podcasts from within the Music app. Perhaps better memory management and multitasking will provide resumed playback more consistently, but it seems that a podcast client (or listening via Safari) will be required.