I had this very same issue, after some research on the Apple Discussions forums I found out that it was due to either the docking connector at at the bottom or the headphone port either needing to be cleaned or that one or both may have been damaged. If you have tried using headphones and can adjust the sound with headphones in this would lead me to believe that this is indeed the most likely scenario.
In my case it was likely to be water related (as my flatmate didn't screw the top on the water bottle in my gym bag and the water got in my phone). If it has been water damaged then you might be able to get some of the water out using silica gel (available on Amazon). Place the silica gel around the powered down phone and seal tightly and leave for at least 48hrs. This should absorb some of the water.
If the damage hasn't be caused by water, you could try cleaning the headphone port and dock connector. You can use a soft bristle toothbrush for the dock connector, again while the phone is off, and give it a decent clean, being careful not to press too hard. You could also use some alcohol here but be careful not to use too much.
The headphone connector can be cleaned using a Q-Tip and alcohol while the phone is off. It needs to be left to dry before turning it on again but it should clean up the headphone port nicely.
If none of this works then it is likely that there is something else wrong with the phone and it would probably be best to take it to Apple, or a third party repair company for them to have a look at it.