I would like to call a command line utility in Mac OS X 10.8 that gives me the ability to convert a text file saved in standard Western Mac OS Roman encoding to the more generic UTF-8.
I will be calling the utility from an AppleScript that I have created. AppleScript is extremely slow when working with very large text blocks. As such, I want to do my text parsing and conversion using the OS X command line. I have found a tool called, "sed", which allows me to do the text parsing. However, there are still many elements of the file that need to be cleaned up, characters that appear as garbage if the file is opened as utf-8 (e.g. smart quotes and ellipses).
I am thinking that forcing a text encoding conversion may help to eliminate all non-utf8 characters in the file. However, I cannot see how "sed" can easily convert the text encoding.
I will have already saved the temp txt file, as MacRoman, to disk using the built-in AppleScript routines.
Do any of you have any ideas as to a built-in command-line tool that can convert text encoding? Command-line for performance and built-in, since other users of my script won't necessarily have the proper toolset if it's not built-in.
Thanks for your help!
