Clipboard and class issue when running AppleScript "do shell script..."

Question

I have a simple bash command that I have tried to envoke from AppleScript but it makes a strange unicode substitution.

Example content of the clipboard:

# 6C (d1/s2): F 11-12 100 Fly (1:12.95Y)

The terminal command:

$ pbpaste | gsed 's/\x2D/\xE2\x96\xB8/g' | pbcopy

changes "-" to "▸" when it finds it on the clipboard

-
HYPHEN-MINUS
Unicode: U+002D, UTF-8: 2D

▸
BLACK RIGHT-POINTING SMALL TRIANGLE
Unicode: U+25B8, UTF-8: E2 96 B8

Output from clipboard is good:

# 6C (d1/s2): F 11▸12 100 Fly (1:12.95Y)

The AppleScript as I wrote it:

do shell script "pbpaste | gsed 's/\\x2D/\\xE2\\x96\\xB8/g' | pbcopy"

Output from clipboard is incorrect:

# 6C (d1/s2): F 11‚ñ∏12 100 Fly (1:12.95Y)

I tried messing around with something like this in AppleScript

set the clipboard to «class ktxt» of ((the clipboard as text) as record)

I also took a peek at the contents of the clipboard after copying the text from a website using this command in terminal:

$ osascript -e "the clipboard as record"
osascript -e "the clipboard as record"
«class HTML»:«data HTML3C6D65746120636861727365743D277574662D38273E3C7370616E207374796C653D22636F6C6F723A2072676228302C20302C2030293B20666F6E742D66616D696C793A20417269616C2C2048656C7665746963612C2073616E732D73657269663B20666F6E742D73697A653A20313270783B20666F6E742D7374796C653A206E6F726D616C3B20666F6E742D76617269616E742D6C69676174757265733A206E6F726D616C3B20666F6E742D76617269616E742D636170733A206E6F726D616C3B20666F6E742D7765696768743A203430303B206C65747465722D73706163696E673A206E6F726D616C3B206F727068616E733A20323B20746578742D616C69676E3A202D7765626B69742D6C6566743B20746578742D696E64656E743A203070783B20746578742D7472616E73666F726D3A206E6F6E653B2077686974652D73706163653A206E6F726D616C3B207769646F77733A20323B20776F72642D73706163696E673A203070783B202D7765626B69742D746578742D7374726F6B652D77696474683A203070783B206261636B67726F756E642D636F6C6F723A20726762283233392C203235302C20323533293B20746578742D6465636F726174696F6E2D746869636B6E6573733A20696E697469616C3B20746578742D6465636F726174696F6E2D7374796C653A20696E697469616C3B20746578742D6465636F726174696F6E2D636F6C6F723A20696E697469616C3B20646973706C61793A20696E6C696E652021696D706F7274616E743B20666C6F61743A206E6F6E653B223E23203643202864312F7332293A20462031312D31322031303020466C792028313A31322E393559293C2F7370616E3E», «class utf8»:# 6C (d1/s2): F 11-12 100 Fly (1:12.95Y), «class ut16»:# 6C (d1/s2): F 11-12 100 Fly (1:12.95Y), string:# 6C (d1/s2): F 11-12 100 Fly (1:12.95Y), Unicode text:# 6C (d1/s2): F 11-12 100 Fly (1:12.95Y)

The "same" text after it has been run through the terminal command and pasted into a Stickies note:

$ osascript -e "the clipboard as record"
osascript -e "the clipboard as record"
«class RTF »:«data RTF 7B5C727466315C616E73695C616E7369637067313235325C636F636F61727466323531330A5C636F636F61746578747363616C696E67305C636F636F61706C6174666F726D307B5C666F6E7474626C5C66305C666E696C5C666368617273657430204D656E6C6F2D526567756C61723B7D0A7B5C636F6C6F7274626C3B5C7265643235355C677265656E3235355C626C75653235353B5C726564305C677265656E305C626C7565303B7D0A7B5C2A5C657870616E646564636F6C6F7274626C3B3B5C6373677261795C63303B7D0A5C706172645C74783536305C7478313132305C7478313638305C7478323234305C7478323830305C7478333336305C7478333932305C7478343438305C7478353034305C7478353630305C7478363136305C7478363732305C7061726469726E61747572616C5C7061727469676874656E666163746F72300A0A5C66305C66733232205C636632205C436F636F614C69676174757265302023203643202864312F7332293A20462031315C7563305C75393635362031322031303020466C792028313A31322E393559297D», «class utf8»:# 6C (d1/s2): F 11▸12 100 Fly (1:12.95Y), «class ut16»:# 6C (d1/s2): F 11▸12 100 Fly (1:12.95Y), string:# 6C (d1/s2): F 11, Unicode text:# 6C (d1/s2): F 11▸12 100 Fly (1:12.95Y)

Just extra info. The characters that were substituted for what I want have this unicode info

ñ
LATIN SMALL LETTER N WITH TILDE
Unicode: U+00F1, UTF-8: C3 B1
∏
N-ARY PRODUCT
Unicode: U+220F, UTF-8: E2 88 8F

EDIT: It seems that piping the results to pbcopy, while working in terminal, was an issue when called from AppleScript.

I changed the AppleScript to:

do shell script "pbpaste | gsed 's/\\x2D/\\xE2\\x96\\xB8/g'; pbcopy"

?? The results show up inside the AppleScript Result window, but this clears the Clipboard... Does anyone know why AppleScript is clearing the clipboard when using pbcopy in a do shell script command?

Answer 1

If you are just trying to replace the - with ▸, then here is how I'd do it:

Note that in the example AppleScript code the first and last lines of it are for testing to actually set the expected content of the clipboard to work with, obviously they are not needed in the working code.

Example AppleScript code:

set the clipboard to "# 6C (d1/s2): F 11-12 100 Fly (1:12.95Y)"

set cbText to (the clipboard as text)

set AppleScript's text item delimiters to "-"
set cbText to text items of cbText
set AppleScript's text item delimiters to "▸"
set cbText to cbText as text
set AppleScript's text item delimiters to ""

set the clipboard to cbText

return (the clipboard)

Result:
# 6C (d1/s2): F 11▸12 100 Fly (1:12.95Y)

---

On a side note, the issue appears to be with the | pbcopy segment of the command.

Here is another workaround:

Assuming # 6C (d1/s2): F 11-12 100 Fly (1:12.95Y) is what's on the clipboard, then:

set the clipboard to ¬
    (do shell script "pbpaste | gsed 's/\\x2D/\\xE2\\x96\\xB8/g'")

Results in # 6C (d1/s2): F 11▸12 100 Fly (1:12.95Y) being on the clipboard.

Answer 2

While @user3439894 did present a workaround solution I did find the answer to the pbcopy issue with help from a user on Reddit. Hope this provides help at some point in the future.

Piping to pbcopy is the correct approach. Since do shell script does not run your login shell (see Technical Note TN2065), you have to supply the locale as an environment variable. For Canadian English:

do shell script "pbpaste | gsed 's/\\x2D/\\xE2\\x96\\xB8/g'| LANG=en_CA.UTF-8 pbcopy"

or

For current system language:

do shell script "pbpaste | gsed 's/\\x2D/\\xE2\\x96\\xB8/g'| LANG=" & user locale of (system info) & ".UTF-8 pbcopy"

This caused me quite a lot of confusion when I was making some clipboard-modifying handlers. And for reference, I found that you're much better off using pbpaste/pbcopy than using the built-in commands from StandardAdditions as these have their own quirks.

So making a small adjustment to their solution results in:

do shell script "pbpaste | gsed 's/\\x2D/\\xE2\\x96\\xB8/g' | LANG=en_US.UTF-8 pbcopy"