I find that when I click "update all" in my iPhone 4S the downloads seem to start and then when they are completed, it stays stuck once the apps are fully downloaded.
I update through wifi, so it's not due to data restrictions.
The blue circles with the "stop" sign inside stayed like this for more than one hour. In the end I stopped them all and had to start and launch each update manually, which isn't ideal, and what I always end up doing.
PS: This is iOS 9.1