I haven't used iMovie specifically but I have used Final Cut Pro X. And AFAIK they source media the same way.
Generally, it works excellently if you keep the following things in mind:
- The external drive needs to be "fast enough" (Basically: USB 3 or Thunderbolt)*
- To save power most external HDDs will enter sleep mode after a certain amount of inactivity. When that happens, it'll generally take a couple of seconds for OS X to wake the drives when you start poking around in iMove again.
(Optionally, HDD sleep can be disabled in System Preferences > Energy Saver but unchecking Put hard disks to sleep when possible)
USB 3 should be totally fine for you.
"In the old days" (or on older Macs), when using USB 2 drives, you'd run the risk of not being able to read/write fast enough. But with your use-case and a modern Mac, that shouldn't be a problem.
The main rule: In general, the faster the drive, the better. This means Thunderbolt over USB 3 and USB 3 over USB 2 and so on. Opting for 7200 RPM instead of 5200 RPM is also a great choice. In the end it comes down to how much you're willing to spend.
I can't point to an official source but Apple-Certified trainer Larry Jordan has a great article that explains the above points in detail. Also, I work at media company so I have a bit of experience on this.
*I could throw numbers at you but since you mention "camcorder" and "iPhone" as sources, I'm guessing that telling you the number of ProRes 422 streams each connection type can handle, would be immense overkill :)