I have enabled email notifications in Server.app (2.2.4) on Mac OS X server (10.8.5)
If port 25 is not being blocked at the local firewall or the ISP, this generally works ok.
I've found that if it is being blocked by the ISP, you can go the mail settings within server.app, and set mail to route all mail traffic through ISP and specify your ISP's SMTP gateway, even if you don't actually have the mail server enabled and/or use the mail server.
The problem that I've run into is that I have a client who uses an email hosting provider that requires a valid MX record for the domain that an email is coming from.
Their email@example.com where they want to receive their Mac OS X server alerts cannot receive the alerts because the messages are seen as coming from an invalid domain with no MX record.
The Mac OS X server is called macminiserver01.local or macminiserver01.private and when the email alerts are sent out from server.app, the from address becomes firstname.lastname@example.org or email@example.com.
Is there a way to manually change the from address so that it is valid?