Automator Shell Script - grep and variables - extracting and combining text

Question

I'm trying to create an Automator Folder Action that does the following when multiple files are moved into it. Each file has the same format (TextA is always the same; StringA and B are different for each file). But the data I want is also duplicated in each file, and I only want one instance of it:

File1.txt

TextAfile1 StringAfile1
TextBfile1 StringBfile1
TextAfile1 StringAfile1
TextBfile1 StringBfile1

An actual example of the text would be:

File1.txt

The delivery of TitleA
The barcode for this is 1234
The delivery of TitleA
The barcode for this is 1234

File2.txt

The delivery of TitleB
The barcode for this is 5678
The delivery of TitleB
The barcode for this is 5678

My automator flow is this so far (Folder action receives files added to Folder):

1. Run Shell Script (display 2 single lines of text starting with "TextA" and "TextB")

grep -i 'The delivery of' "$@"
grep -i 'The barcode' "$@"

2. New Text File (plain txt)

3. Loop through all files (automator seems to do this automatically)

4. Run Shell Script (Delete "TextA" and TextB from the beginning of all lines so they all start with the text StringA that immediately follows)

   sed -e "s/The delivery of //g" "$@"
   sed -e "s/The barcode for this is //g" "$@"
   

5. Run Shell Script (alphabetize sort)

   cat "$@" | sort
   

The resulting Text file should look like this:

TitleA 1234
TitleB 5678
...
TitleZ ####

Right now I'm getting this (without quotes):

"/users/path/to/file1.txt:The delivery of TitleA"
"/users/path/to/file1.txt:The delivery of TitleA"
"/users/path/to/file2.txt:The delivery of TitleB"
"/users/path/to/file2.txt:The delivery of TitleB"

"/users/path/to/file1.txt:The barcode for this is 1234"
"/users/path/to/file1.txt:The barcode for this is 1234"

"/users/path/to/file2.txt:The barcode for this is 5678"
"/users/path/to/file2.txt:The barcode for this is 5678"

I'm trying to delete the path, delete the preceding text, and remove the duplicates. So ideally I'll end up with:

TitleA 1234
TitleB 5678

Bonus would be to also sort the final txt file alphabetically. But I could settle for pasting it into Excel and sorting there etc. although sorting in script would be better if not too dificult.

Am I on the right track at all? I've tried all different combinations of this workflow and it feels like there's a fundamental flaw with it.

Answer 1

sed is not very well suited for multiline processing, but it's easy to accomplish with awk:

$ awk '/^The delivery of/ { title = $4 }
       /^The barcode for/ { print title, $6 }' File1.txt File2.txt | sort -u
TitleA 1234
TitleB 5678

To integrate this into a Folder Action, create an Automator Folder Action workflow with one "Run Shell Script" action.

You didn't describe what you want to do with the output, so either capture it with an additional action or add > ~/my-output.txt to store it in your User directory.

Answer 2

The operating system comes with lots of small tools that manipulate text. In this example we chain rev (reverse), cut, paste, and sort together with pipes.

rev file1.txt file2.txt | cut -d' ' -f1 | rev | paste -d' ' - - | sort -u

• rev reverses each line of text in the files
• cut grabs the first "field" of text on each line
• rev again reverses the text from the output of cut
• paste glues every two lines into a single line
• sort removes duplicates

This solution is only a demonstration and would be better done with awk or perl

Answer 3

I think that you're seeing those results because you are working with the text files in addition to their contents.

I have no interest in working with folder actions but if you add the following actions to a new folder action they should work with text files when they're added to the folder.

• Combine text files - combines the eight paragraphs from the two files.

• Run shell script - sed -e 's/The delivery of //g' -e 's/The barcode for this is //g' Removes the two specified strings

• Run shell script - sort -u Sorts results and removes duplicate lines

• Run applescript - Pairs the title with the barcode

on run {input, parameters}
    set paraList to items of input
    set paraLength to ((length of paraList) / 2)
    set output to {}
    repeat with x from 1 to paraLength
        set end of output to item (x + paraLength) of paraList & space & item x of paraList
    end repeat
    return output
end run
--> {"TitleA 1234", "TitleB 5678"}

• New TextEdit Document - should deposit desired text in new document

TitleA 1234
TitleB 5678

What it does:

The first action gathers the text (as paragraphs) from the files added. The first shell script runs two sed commands which remove your unwanted strings. The second shell script sorts the remaining text and its -u option removes any duplicate lines.

The run applescript action basically breaks the items into two groups, loops through them and pairs the titles and barcodes. Finally, the results are deposited in a new text document. If you drop additional documents at once that follow the same format, then they should be reflected in the results.

NB for testing purposes, I used a workflow containing the above actions but with the following at the top to feed 'file1.txt', etc… to the 'combine' action.

• Get selected finder items - the two (or more) text files