Ask Different is a question and answer site for power users of Apple hardware and software. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a million+ files spread across four fiber-connected arrays. I need to get the path, filesize, and last_modified date to insert into a mysql database. What would be the best way to do this?

Here is what I am currently doing:

sudo find "$FULFILLMENT" "$ARCH1" "$ARCH2" "$MASTERING" -type f -exec ls -lT {} + > $FILE

And then I parse that file (using Python) to insert it into my db.

What would be the best way to accomplish the above?

share|improve this question
Is there anything wrong with what you are currently doing? Also, I don't see how we can answer this properly without some insight on which files and on what location you need to search. – Gerry Feb 1 '13 at 22:52
There are a millions files spread across four volumes. They're all types of files (really, it's four different computers with 100's of TB storage each). What I'm currently doing takes about 3 hours, I was wondering if there's something that is faster. – David542 Feb 1 '13 at 22:53
And how much of those 3 hours does the find command take to complete? In any case, I do believe find is the most appropriate tool for listing the files. – Gerry Feb 1 '13 at 22:55
The find command takes about 99% of the time. – David542 Feb 1 '13 at 23:00
You could try find -type f -ls, though I'm not sure that will make any significant difference. It sounds like there just is a lot of data to process. – Gerry Feb 1 '13 at 23:04

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.