I have noticed that the Finder's search misses many times matches from PDF documents even in strict Folder search.

It would be very nice to get information about possible skipped matches. One skipped matched is like 100 minus points. Adobe search seems to be the must use here otherwise.

Zhora's answer reminds me about the problem in hand. The default search utilities of OSs are unstable because of variety of different filetypes. I personally produce Tex files so no problem in indexing plain tex and searching fast. However, my colleges produce WYSIWYG documents with PDF output. This thing creates a demand for search programmatically.

One necessary approach in the effectiveness of search is to limit the search space. I like chflags and .noindex extension in doing this.

How can you measure the effectiveness of Finder's search by Python or other language?

  • Just wondering! what good will that do ? – Ruskes Jul 1 '14 at 11:13
  • Sometimes one missed result is much more expensive than the rest of the results. If there are skipped entries, I must know. Such search is very unstable. – Léo Léopold Hertz 준영 Jul 1 '14 at 13:51

Over years of using OS X search tools (Spotlight, Find, Smart Folders), all of which, I believe, use the collected file metadata in various ways and having gotten a fairly good idea what various combinations of search terms might find I am still often unable to successfully retrieve every file I think my search should find. At one point I began to systematically test various searches to try to determine some basic patterns so I could be certain that the resulting files contained every (or even "the specific") file I was looking for. I found that there were so many things that could subtly affect a search (such as word order, capitalization, differences between the use of numeric and alphabetic characters, using partial words that matched the beginning of piece of metadata versus a substring that would match within a the same attribute, quotes and double quotes, unmatched quotes, the various logical operators [AND, OR, NOT or &, |, ^ or &&, ||, or some or all of these so I can never quite remember the right form to use], the various "names" for pieces of metadata such as "date:" or "kind:" or many others, the complete list hidden away from most users abilities to find)…well, basically I learned a few things I could count on but I wonder if there is anyone who could actually predict the exact results of all but the most basic search. And then I’ve run basic searches that produce lists of files that I’m sure are missing some files that should be found or missing others that should be there and yet I’m not sure why. Sometimes after time spent staring at the results I can see what happened; sometimes I never do.

All that is to say I am also bothered by inconsistencies for certain types of searches that I want or want to depend on being complete (as I think you are looking for yourself). On the other hand, these same search tools are so essential to the ease of use of my Mac that I’d truly miss them if they were unavailable (at times I’ve been amazed at what I could find such as older pieces of my writing that were in files which had lost their filenames due to accidental deletion yet with a few key words could be quickly located.) I have also watched people I know looking for files on a PC running Windows and think, "why don’t you just search for it?" Then I recall how I often avoided using the files searching facilities when Windows was my main OS.

I believe that Apple’s search facilities will probably not be satisfactory to what you want, at least when used "intuitively." That said, there are different commands that can be run using terminal that may allow you to get better results in your case, but that can only be determined by trial and error. Their value, though, can be in providing ways to see exactly what metadata exists for some file which you feel is left out of the results of a search and whether the desired type of data is missing or a particular matching type of data is missing or otherwise. The various system commands are those like mdutil and mdls and possibly other related commands. I usually run man for one of these, such as man mdls and then refer to the related commands at the end of the manual page under the SEE ALSO heading. The difficulty is that the learning curve can be steep depending on how far you wish to delve into the various search techniques. Often the hardest aspect is not in knowing how to run a type of search but more in being able to find out the various names of the search keys to search on, which may require running other md… commands to dump certain keys and/or values of the system metadata.

Along with all of the basic unknown factors there is always the possibility that the metadata store is corrupt or not complete and needs to be rebuilt. And then, too, the metadata is extracted from files as they are created or changed by various "modules" each of which handles some particular type or types of metadata extraction. Many of these modules are provided by Apple and many come from various vendors to allow different types of files to be processed and different attributes collected. So problems my exist in the metadata collection just as they might in the search facilities. That’s where some system commands can help one to see just what is wrong with the files metadata or a particular query string.

If you are basically interested in searching for PDF files and the Adobe search features meet your needs I’d probably suggest going with those (or perhaps other third party search tools that may have a more defined and predictable search methodology). And use Apple’s search facilities for what they can help with (and in many cases, despite possibly not returning every desired file, can produce "fuzzier" results that can help to form a better query for use in an alternate search engine).

Sorry to go on without being able to give more useful ideas but I hope some of what I’ve said helps you to decide upon a method that works for you. (I love my Mac…most likely will always use a Mac…but sometimes I think that Apple doesn’t document parts of OS X just so it’s never possible to point at something and say it doesn’t work right, since it might work exactly as it was intended. You never actually know for sure!)

  • This reminded me about a programmer who always uses Python to do simple searches in his computer (either OSX or Windows). It just works. He says that default search tools of OSs are unstable (like we see). However, neither, he had developed a tool that can measure the effectiveness of these default searches in OSs. I am writing my documents as Tex -files which have only plain text and can therefore be indexed fast. So the decision to have good search depends on how you work. I produce Tex files but colleges - WYSIWYG files and thus PDF files. So Python seems to be good tool here for dev. – Léo Léopold Hertz 준영 Jul 2 '14 at 13:18
  • This is actually closest to the Python search which my friend does stackoverflow.com/questions/2186525/…, in older Pythons however with os.walk. – Léo Léopold Hertz 준영 Jul 2 '14 at 13:45
  • I opened a more general question about this here stackoverflow.com/questions/24533029/… since the problem is not that different in other OSs. – Léo Léopold Hertz 준영 Jul 2 '14 at 13:55
  • Actually, I managed to improve my local search many folds by discovering noindex apple.stackexchange.com/questions/148850/…. To have too big index with bad documents makes any search bad. So limiting the space is a good way to go. I like mdls to see things. – Léo Léopold Hertz 준영 Oct 8 '14 at 20:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .