Have you tried using telnet to diagnose the connection? Issue the command:
telnet <hostname or ipaddress> 13
Insert either the hostname or ip address of the time server you wish to connect to. Be sure to include the "13" that follows. This is the port number that NTP communicates on.
I tried it with the time server you specified above and here are the results:
telnet 18.104.22.168 13
Connected to utcnist2.colorado.edu.
Escape character is '^]'.
57126 15-04-14 11:48:04 50 0 0 907.8 UTC(NIST) *
Connection closed by foreign host.
The server will report that the connection was closed by the remote host; this is perfectly normal. However, you can confirm that everything is working by noticing that the server sent the time string (it's the line that starts with 57126 in the example above.
If this test works, then you know it isn't your firewall, router, or ISP. That means we have to look at the NTP service running on your Mac.
If it dosen't work, chances are, something is being blocked. Try turning off the firewall on both your mac and router (just temporarily for diagnostics). Does it work now?
Re-enable the router firewall and re-check. Still working? If not, your router is blocking the traffic. If it is, re-enable the firewall on your mac. If that breaks it, then you know that your mac was blocking it.