If I were to make an iOS app that whichs primary function is to play specific 3rd party YouTube videos, is that okay to do without asking for their permission? Or do I need to have a legal agreement with each publisher?
There is a setting in youtube called public.
That means anyone can view it, including downloading or from other apps ect.
If the owner wants to prevent that he/she can do that in youtube settings.
As long your app is not breaking the youtube T&Cs and you are fine.
Consider following: many websites contain link to youtube and many even allow embedding the videos, so it is up to the owner to allow or not.