Move folders using Terminal by referencing a CSV file

Question

I have a CSV file containing the following two columns:

1. Column A: From Directory (e.g. /Volumes/Backup Plus/Salesforce/Order Test/OR-1000049439).

2. Column B: To Directory (e.g. /Volumes/Backup Plus/Salesforce/Opportunity Test/Q-DNYE20180508-2308).

I need to write a script in Terminal that will loop through the CSV and move the directory from Column A into the directory from column B. The end result would be the OR-1000049439 folder residing inside the Q-DNYE20180508-2308 folder). There are roughly 13,000 rows in which I need this to occur.

Can someone please help me on how to do this?

Here is a copy of the first few rows in the CSV file:

I need the folder in A2 to end up in the folder on B2. I need the folder in A3 to end up in the folder on B3, etc...

Brians-MacBook-Pro:Data Migration briangannuscio$ Head OrderFileV2.csv
/Volumes/Backup Plus/Salesforce/Order Test/OR-1000050081,/Volumes/Backup Plus/Salesforce/Opportunity Test/Q-RGORD20170207-1526

Brians-MacBook-Pro:Data Migration briangannuscio$ File OrderFileV2.csv
OrderFileV2.csv: ASCII text, with CR line terminators

Answer 1

There are many ways to tackle this task. Below is an approach using the Go language. I expect others will soon be along with alternative tools and languages.

If this is a one-time task, expect to spend time handholding the process.

If this going to become a regular automated task, spend time now to get the script or tool right.

Things to be aware of:

• Appreciate that the csv file format is not fully standardized.
• Know that Excel has a reputation with the csv format.
• Does any file or folder name contain a quote, space, or comma?
• What happens if folders are listed but missing on disk?
• What happens if a folder for moving is listed twice?
• What happens if a folder for moving already exists at the destination?

Back-up your data before running any answers.

csv-move

The csv-move tool requires UNIX/LF line endings. Converting the line endings is a problem solved elsewhere.

Ready to use binaries of csv-move are available in the latest release. Below is an extract of the code:

// Read from stdin - expects LF line endings
r := csv.NewReader(os.Stdin)
r.FieldsPerRecord = 2

for {
    line, err := r.Read()
    // ...

    // Perform the command
    cmd := exec.Command(mvPath, "-vn", line[0], line[1])
    if err := cmd.Run(); err != nil {
        log.Fatalf("move failed: %s", err.Error())
    }
}

Test your CSV file can be parsed as two entries per line:

./csv-move < input.csv

Once tested, and you are happy with the printed move commands, perform the move:

./csv-move -move < input.csv

Answer 2

You should expect that any csv file will need to be sanitized before using it in the shell environment. At minimum, the header, if it exists, should be removed, all line terminating characters will need to be converted to LF (the linefeed character), and the last line of the csv file will need to be terminated.

The file command shows that your csv file has CR (carriage return) terminated lines. So, using simple available commands, you can convert the CR characters to LF characters with tr then pipe the results to sed to remove the header and terminate the last line, creating a new sanitized file.

tr '\r' '\n' < OrderFileV2.csv | sed '1d;a\' >sanitizedOrderFileV2.csv

The new file should be checked for contiguous rows. Empty fields and lines should be removed. You now should have a file that you can feed into a while read loop.

Answer 3

On shell level

while IFS=\, read -t $'\r' s t; do
    echo mv -n "$s" "$t/"
done < orderfile.csv

Remove the echo once you are sure the result looks ok.

As a small safeguard against errors I would recommend to limit a CSV to about 1000 lines.