HELP!!! Trying to split a csv file admin_bids_view.csv into multiple files. The CSV has 2000 rows and 7 columns. I would like a new file per ID number in column 1.

Example data below:

enter image description here

Have tried awk -F\| '{print>$1}' admin_bids_view.csv

But the below is being returned.

awk: can't open file admin_bids_view.csv source line number 1

Please can someone point me in the right direction?

Thank you.

  • Is the file in the current directory, the one you want to process? The error indicates that the file is missing. – nohillside Nov 22 '18 at 14:22
  • It's probably not. How do I do this? Thank you. – SJB Nov 22 '18 at 14:32
  • You need to store your CSV into the same directory your shell uses as current directory. Or cd into the directory your file is in – nohillside Nov 22 '18 at 15:27

Your approach would mostly work, although you'd end up with column A in the output file, which may not be what you want. Here's another approach that doesn't put column A in the output file:

awk -F, '{outfile=($1 ".csv") ; print substr($0,index($0,$2)) >>outfile ; close(outfile)}' <name_of_input_file

You said "CSV", so use a comma as the field separator.

  • Thank you. That is exactly what I need it to do. I got the following error: awk -F, "{print substr($0,index($0,$2)) >$1}" admin_bids_view.csv awk: syntax error at source line 1 context is >>> {print(-bash,index(-bash,) <<< awk: illegal statement at source line 1 awk: illegal statement at source line 1 – SJB Nov 22 '18 at 16:57
  • Hm. Replace the double quotes with single quotes and try it. I tested my answer, but on a Windows machine. ^_^ – Marc Wilson Nov 22 '18 at 17:25
  • That worked, thank you. The next problem encountered is: awk: 263 makes too many open files Is there an easy solution to this. Sorry for all the questions. – SJB Nov 22 '18 at 18:29
  • Yeah, your problem is actually two related problems. You need awk to keep each file it creates open so that it can append to it, and keeping the file open eventually exhausts the available file descriptors (there must be a LARGE number of unique values in column A). I've edited my original answer... try the edit and see if it works. Bear in mind that since this is now using an append, you need to erase the result files before you run it. – Marc Wilson Nov 22 '18 at 19:42
  • You are an absolute lifesaver, that worked perfectly. Final question, I promise, is there a way to add .csv to the file names? Thank you. – SJB Nov 22 '18 at 20:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .