On my iPhone 5s I see a symbol for headphones next to the battery usage symbol. This only appears when the headphones are NOT plugged in. Also I am unable to get a sound notification for mail or messages. I think these are related problems.
The headphone symbol is shown when you are connected to Bluetooth headphones/speakers, which would be true if you have a Bluetooth device auto-connecting (which would not be used when you plug in headphones). Go to Settings → Bluetooth and see which device you are connected to, and disconnect if you wish. Sound notifications will be routed to this device so that is why you do not hear them.
If you no longer want to use Bluetooth to connect to these particular headphones/speakers (i.e. you only want to use a cable) you can also forget the device in the same settings by tapping on the info symbol and selecting Forget this Device. This will stop your iPhone from auto connecting to them. If you decide in the future you'd like to use Bluetooth again, you can pair your headphones to your iPhone again later.
I have iphone 6S Plus, IOS 10.2. I have digital hearing aids that allow me to stream my music from my phone, through a Bluetooth remote into the receivers in my ear. If I just turn off the remote the phone will stay in headset mode. I have to change it back to music directly from iphone before turning off the remote unit..