Following Monomeeth's answer, I pressed the Sleep / Wake button together with Volume down and the iPhone restarted, and the horizontal bar moved back 20% and worked its way up to 99% and stopped there for another hour. And I repeated this 2 times already and it was doing the same thing.
And then, I went to an Apple Store, and was told that I can try 2 things.
Use a Mac to connect to the iPhone and use Recovery Mode, and try
- to upgrade it from the computer and hope to keep the data on the phone,
- restore it to factory setting, and lose all data on the iPhone.
So we tried (1) and it is the same result of 1% till finish and it got stuck. So it looks like I have to lose all data.
But there is a third choice: use an old iPhone first, and wait for iOS 13.0.1 or iOS 13.1, and see if the upgrade using the Mac can successfully handle it. My Photo Album on the iPhone was corrupt one time and the iPhone rebuilt the albums. I wonder if the upgrade to iOS 13 is related to this issue (the upgrade cannot handle the photos to rebuild the index for iOS 13, and it could not go on).
I waited for iOS 13.1 and went into iPhone Upgrade from Macbook, and the weird thing is, the Macbook was downloading iOS 13.1, and after about 15 minutes, the download wasn't finished yet, but the iPhone went onto upgrade itself using iOS 13.0.
So I had to wait for the download to complete, and then go again into iPhone Upgrade for iOS 13.1.
And it didn't fix the problem. The upgrade again stopped at about 99% of the grey horizontal bar.
And then, I waited for iOS 13.1.1, and repeated. And it failed.
And then, I waited for iOS 13.1.2, and repeated. And it failed.
So I finally "restored" it to the latest iOS, which is 13.1.2, and it could restore and boot up this time. But I had to download everything and sign in again and set up everything.
I wasted about 8 hours total and lost almost all info on the iPhone. Sure, I should have backed up the iPhone regularly, but I wouldn't expect an iOS upgrade cause my phone not able to even boot up, and let me try and try again, with the final way to restore it to "factory".