Mavericks Server - nothing enabled, but services still running and logging messages

Question

I have a Mavericks server with ALL services disabled in Server.app in an attempt to figure out what's causing a huge number of log messages. When I check to see what services are running, I see the output at the end of this question.

It implies that the Wiki server (com.apple.collabd), the Xcode server (com.apple.xcs* and com.apple.servermgr.xcode.*), the calendar server (org.calendarserver.*), Profile Manager (com.apple.DeviceManagement.*) and others are all running, or at least parts of them are running.

The main concern is thousands of messages per minute that look like this:

Jan 18 02:59:25 domain.com collabd[84733]: [CSODService.m:78 f0a3000 +0ms] Failed to convert user externalID servermgr_xcode to uuid_t for membership check
Jan 18 02:59:25 --- last message repeated 1 time ---
Jan 18 02:59:25 domain.com serveradmin[44306]: [XcodeRequestHandler+DeviceSupport.m:183 73552310 +0ms] Filtering device because it does not have a serial number (e.g. mac or simulator device)
Jan 18 02:59:25 domain.com collabd[84733]: [CSODService.m:78 f0a3000 +0ms] Failed to convert user externalID servermgr_xcode to uuid_t for membership check
Jan 18 02:59:25 --- last message repeated 1 time ---
Jan 18 02:59:25 domain.com serveradmin[44306]: [XcodeRequestHandler+DeviceSupport.m:183 73552310 +0ms] Filtering device because it does not have a serial number (e.g. mac or simulator device)

These appear to be from the disabled Wiki and Xcode services. They get annoying and also cause excessive log file I/O, not to mention other effects of running services that aren't needed or wanted.

My question is threefold:

1. Why are these running if they're disabled in Server.app?
2. Is there a thorough list of server processes and descriptions, preferably provided by Apple?
3. How do I disable them, and which ones are critical to keep running?

I'm specifically looking at ones like SCEPHelper for which there is no manpage and doesn't seem to have any description online. Certain ones, like apspd, are used for server push notifications and so should stay running. I'm mainly concerned with which ones must stay running for things like remote administration. I've already tried serveradmin stop X for the Xcode and calendar services, and although I can stop and start them that way, those few stray services still stay running.

Also, a bonus question: where is the manpage or help document for serverctl?

Disabled and enabled services shown here:

$ sudo serverctl list
{
    disabledServices =     (
        "com.apple.AccountsConfigService",
        "com.apple.afctl",
        "com.apple.AssetCache",
        "com.apple.collabd.expire",
        "com.apple.collabd.notifications",
        "com.apple.collabd.preview",
        "com.apple.collabd.quicklook",
        "com.apple.collabd.scm_poller",
        "com.apple.collabd.stats",
        "com.apple.DeviceManagement.dmrunnerd",
        "com.apple.DeviceManagement.php-fpm",
        "com.apple.ftpserver",
        "com.apple.list_server_mgr",
        "com.apple.mail_exclusion",
        "com.apple.mail_status_init",
        "com.apple.mail_status_periodic",
        "com.apple.ppp.l2tp",
        "com.apple.ppp.pptp",
        "com.apple.push_notify",
        "com.apple.Rooms",
        "com.apple.salearn",
        "com.apple.saupdate",
        "com.apple.server.filesharing",
        "com.apple.server.mail.alerts",
        "com.apple.server.netboot",
        "com.apple.server.opendirectory",
        "com.apple.swupdate.host",
        "com.apple.swupdate.sync",
        "com.apple.xcs.stats",
        "com.apple.xcsbuildd",
        "com.apple.xcscgsessiond",
        "org.amavis.amavisd",
        "org.amavis.amavisd_cleanup",
        "org.calendarserver.calendarserver",
        "org.clamav.clamd",
        "org.clamav.freshclam-init",
        "org.clamav.freshclam",
        "org.dovecot.dovecotd",
        "org.dovecot.fts.update",
        "org.freeradius.radiusd",
        "org.isc.named",
        "org.jabber.jabberd",
        "org.jabber.proxy65",
        "org.postgresql.postgres"
    );
    enabledServices =     (
        "com.apple.apspd",
        "com.apple.collabd",
        "com.apple.DeviceManagement.devicemgrd",
        "com.apple.DeviceManagement.postgres",
        "com.apple.DeviceManagement.SCEPHelper",
        "com.apple.opendirectorybackup",
        "com.apple.server.alertsd",
        "com.apple.server.eventsd",
        "com.apple.server.v2.stats",
        "com.apple.ServerEventAgent",
        "com.apple.servermgr_xcode.get_currently_connected_devices",
        "com.apple.servermgr_xcode.sync_adc",
        "com.apple.servermgrd",
        "com.apple.disks.smart.status",
        "com.apple.xcscredd",
        "com.apple.xcscredhandler",
        "com.apple.xcsdevicesnifferd",
        "com.apple.xcsxcodeappwatcher",
        "org.calendarserver.agent",
        "org.calendarserver.relocate"
    );
}

Answer 1

Your list of active services mirrors mine on a server with only DNS running per the server app:

enabledServices =     (
    "com.apple.apspd",
    "com.apple.collabd",
    "com.apple.DeviceManagement.devicemgrd",
    "com.apple.DeviceManagement.postgres",
    "com.apple.DeviceManagement.SCEPHelper",
    "com.apple.opendirectorybackup",
    "com.apple.server.alertsd",
    "com.apple.server.eventsd",
    "com.apple.server.v2.stats",
    "com.apple.ServerEventAgent",
    "com.apple.servermgrd",
    "com.apple.disks.smart.status",
    "com.apple.xcscredd",
    "com.apple.xcscredhandler",
    "org.calendarserver.agent",
    "org.calendarserver.relocate",
    "org.isc.named"
);

In your case, the collabd should be running and it is complaining about the machine serial number. Check that your hardware serial number is in fact readable via software:

system_profiler SPHardwareDataType | grep 'Serial'

The best way to troubleshoot server when you don't trust the graphical server app (or want more details) is to use serveradmin command as opposed to the serverctl command. As you have noticed, serverctl is undocumented, so you'll want to stick with the stable serveradmin interface if at all possible.

Here's a short loop command that dumps just the running/stopped status of the various subsystems that serveradmin knows about:

for service in `sudo serveradmin list`
do
  sudo serveradmin status $service | egrep "RUNNING|STOPPED"
done

You could easily change that grep command to stop only the running services...

sudo serveradmin status $service | grep "RUNNING" && sudo serveradmin stop $service

Answer 2

I see the same thing on my server, which is what brought me here. I'm sorry I cannot help much, but to point to this thread with the serverctl command, e.g.

sudo serverctl disable service=com.apple.collabd.notifications

Answer 3

There are a small number of daemons that are required for remote and local administration.

There are also an even smaller number of processes that are so tied into the OS that is easier to leave them running than to turn them off.

Which of the three possibilities any given daemon falls into is almost impossible to know. The thread pointed to in another answer does give you a list of processes that can be safely unloaded by launchctl to go even further back to nothing than either the Server app or serveradmin command line tool. Awhile ago I tested and they are fine to unload.

The better question is why you are getting thousands of errors a minute written to the log. There must be something wrong with your setup as my server does nothing of the sort.

Let's see if we can fix that problem.

Have you tried to set this server up as an Open Directory master yet? I only ask as getting that working means that a number of certificate, DNS and host table issues have to be 100% for an OD master to come up and by the look of that log output there may be a problem with the security on the server being over zealously broken. My other question, in the log you have given does it literally say 'domain.com' or does it give your domain and you changed it for security reasons?