It seems that the CPU throttling will stop once device's battery is replaced.
In a statment to TechCrunch, Apple says they will throttle devices "only when needed."
Last year we released a feature for iPhone 6, iPhone 6s and iPhone SE to smooth out the instantaneous peaks only when needed to prevent the device from unexpectedly shutting down during these conditions.
As this happens "only when needed," replacing the battery should stop CPU throttling from happening. Explaining his experience with the same issue, Forbes contributor Antony Leather says that performance did increase on his iPhone 6 after replacing the battery.
My iPhone 6 was a prime candidate - I had actually replaced the
battery myself a couple of years ago, but I'm a heavy user and already
noticed its longevity had started to decrease again...
Finally, I replaced the battery with a new one from Amazon, charged it
to be well away from the low power mode level and re-ran the test. The
difference was incredible. The single core score rose from 1038 to
1562 - an increase of precisely 50%.
Check out Forbes' Boost Your iPhone Speed By 50%: How To Replace The Battery And Stop Apple Degrading Performance
Check out this excerpt from How-To Geek's You can Speed Up You Slow iPhone by Replacing the Battery, which suggests that replacing the battery will improve performance.
If your phone is indeed slowing down as a result of poor battery health, replacing the battery will give your phone new life. Not only will you get a longer-lasting battery by replacing an old one, but your phone should jump back up to its top speed.