The best way to control text size would be to convert the file to text/markdown. That way you would have total control over the sizing.
It really depends on how proficient you are in editing inDesign or simplifying the formatting there to see if you can tweak things there to isolate the part of the native -> ePub conversion that's running amok.
If you're good there, take a simpler approach to fix things. If you have no aptitude or time to learn inDesign or clean up the document's formatting (I've seen some PageMaker files that got moved to inDesign that are incredibly convoluted and needed to be thrown out and started over from text).
Sorry I can't be more specific. If you want to ask a follow on question with specific details on what item is being converted or can upload a minimal test case, someone might have a better idea on preventing the poor conversion from one format to another.