10

Spotlight, is a fantastic tool. Took some years, but now I use the command line tools and it's awesome. However today, I found that spotlight does not index /System (!) I'd like spotlight to index everything and that way I can cobble together a nifty IDS solution.

How can I index every single thing on my drive? Querying the last modified time would help a lot in security.

EDIT: Simply for reference. 

 mdutil -pEsa -i (on|off) volume ...
    Utility to manage Spotlight indexes.
    -p      Publish metadata.
    -i (on|off) Turn indexing on or off.
    -E      Erase and rebuild index.
    -s      Print indexing status.
    -a      Apply command to all volumes.
    -v      Display verbose information.
NOTE: Run as owner for network homes, otherwise run as root.

EDIT: Upon further investigation, the tool I wanted was radmind

Improve this question
6
  • I think I got it working by forcing it using the root user, mdutil -E / Seems obvious but I'm unsure. I can find ARD agent and all that other stuff now though...
    – chiggsy
    Nov 15, 2010 at 10:23
  • @chiggsy Running spotlight as root is possibly the worst idea you could come up with… really.
    – Martin Marconcini
    Nov 15, 2010 at 15:22
  • @Martín Whyever would you say such a cryptically ominous thing, my dear fellow? Apple instructions are for their best interests, certainly, which at times vary with what I perceive to be my own interests, undoubtedly, but surely you have some reasoning for this statement? An answer perhaps?
    – chiggsy
    Nov 16, 2010 at 5:58
  • 1
    @chig Because it’s a daemon that has the potential to read (and update) metadata for files (last modified time for example). Giving a userland program the access to potentially modify your entire filesystem on a POSIX operating system, is basically considered a bad idea. Spotlight could have a serious security bug or could be compromised in a way where it could simply delete files instead of search them. Do you want your /System folder to be wiped? Of course that has been a little bit extremist (my statement) but I believe that no sane user would ever touch root for anything like that.
    – Martin Marconcini
    Nov 16, 2010 at 10:11
  • 1
    @chigg I cannot give you an answer because I’m not sure if Spotlight running as root will index the whole drive. I don’t know if Spotlight is not simply programmed not to index certain things. I have never really tried and if you want to go ahead and try it, the whole community would be interested in your results. I still consider it’s not a good idea, not because Apple chose not to do it, because I would consider it a dangerous idea if spotlight ran a root and indexed the whole drive, but I’m no security expert nor I work on that field. I’m a programmer.
    – Martin Marconcini
    Nov 16, 2010 at 10:13

1 Answer 1

Reset to default
4

From a great deal of research, and looking around, I have some data on this issue:

By default, spotlight will not index certain folders:

  • /System
  • /usr
  • hidden files or directories.
  • Other user files.

To add a file path to spotlight you can run 

mdimport -r /path  

man mdimport

has the information on that.

Now, since I am aiming for a poor mans IDS out of all this stuff, this desire being driven by the knowledge that Spotlight indexes my drive all the time, which is what would happen anyway with other host based IDS there were some considerations and other tools to involve.

Considerations:

Spotlight will only show you what your user should see

That's what the documentation says. I can see stuff I installed as root, but I can't see my other user. However,I can see /usr /usr/libexec and the /System tree. That'll do.

Hidden files and folders do not show up in the search

This will be good when the RIAA remotely scans your drives for music without proper credentials ( Trust your feelings you know this to be true. ) but is not the greatest news in this case.

To conclude, there is a lot of stuff to do to use this tool effectively. The secret is that Apple signs everything digitally. 

man codesign

will tell you about 

codesign -v file

which should return nothing if the file is unmodified. Note that this is not a checksum but a digital certificate from Apple, so only big money will allow this to be faked.

I of course meant to say that it will be quite secure, and easily detectable if a binary program is changed.

Won't stop everything, but it will allow me to periodically bark

"Did anything just change?" , run a spotlight search on the "kMDItemKind" attribute , pipe it through codesign -v and see if something did change, or search on modification time or whatever.

To address the userland statement above, I can check to see I have the same spotlight ( I have copied codesign to my recovery media. ) intact spotlight means that I can trust it to do it's ordinary tasks. Using the mdimport -r /path is indeed a better idea, since that will exit if run as root.

Certainly there is a question of security here, but as mentioned above, spotlight indexes a bunch of things, and shows you what you should see. Your little sister will not be able to find your collection of late 1990 artistic nudes, nor will you be able to find her secrets, but root should be able to see everything. There is a straightforward system of authorizations in OS X that govern which right a program can have, but since this is virtually unknown to most, they simply type in a password when a box comes up to authenticate something they have downloaded, and it installs as root. A certain search engines software does exactly this. Hell, the system is actually more secure than before, I ran the old python importer, and it failed, because it asked for my admin password and tried to run mdimport -r as root! I had to run it myself.

(Oh it's very nice with the python files, lovely really)

Hope this helps somebody else.

Improve this answer
5
  • I find the RIAA comment on this question a bit weird. It looks a bit like "conspiracy theories". Also, I don't see the relationship between 'codesign' and hidden files. Otherwise, the answer is not bad.
    – Ricardo Sanchez-Saez
    May 14, 2011 at 13:06
  • I live in Canada, so the RIAA thing is "recent history" according to Wikileaks. Quite well documented, came out day before the election, May 2. Spain as well, actually, although they got the news early enough to vote it down. I would have liked a bit more time, but that is a thought from the dead, uncapped, compassionate past. That's over with.
    – chiggsy
    May 14, 2011 at 15:00
  • Yeah, I agree that the RIAA is trying all kinds of nasty things to find users who have copyrighted material on their HD drivers, but I doubt apple would allow this (via providing a remove backdoor to RIAA or similar) on a vanilla Mac OS X install.
    – Ricardo Sanchez-Saez
    May 14, 2011 at 15:28
  • Here is the thing. I want to agree with you. Then I look at the TPP, the son of ACTA. Apple can be depended to stand up where France caved?
    – chiggsy
    May 14, 2011 at 16:06
  • So, here we are 2 months later, the ISP's are policing ( voluntarily, at the suggestion of the White House ) their subscribers, and Jobs made a deal with the labels. Icloud offers 'amnesty'. How I wish I was dead wrong about what I said wrt the RIAA :(
    – chiggsy
    Jul 17, 2011 at 9:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .