4

I have two issues that seem related. When copying text or filenames containing umlauts or e.g. å ä ö, OS X can't seem to handle the characters in a sane way. Apparently this is a well-known problem. For example:

Here is an example of the usual result when copying text from a filename or from inside a pdf in Preview and pasting it in an editor. The first line is the result, the other one is corrected.

example of copy

The difference is clearly visible, since the current font (Courier Prime) doesn't support the first version.

Is there any way to fix this? Alternatively, is there any OS X-service available to "clean up" text or normalize it in the correct way?

Improve this question
2
  • 1
    I don't understand what normalization has to do with your problem, which looks like a font substitution issue.
    – Tom Gewecke
    Mar 1, 2013 at 17:25
  • 1
    PS The reason for font substitution may be that Courier Prime is a somewhat limited font and lacks combining diacritical marks. The copy I downloaded does not seem to have any. That would mean it cannot handle any decomposed accented characters and normalization form C is required. I think the best fix may be to use a more complete Unicode font instead. They all normally include combining diacritics as far as I know.
    – Tom Gewecke
    Mar 1, 2013 at 19:42

1 Answer 1

Reset to default
6

HFS+ requires filenames to be in decomposed form (LATIN SMALL LETTER A + COMBINING DIAERESIS) instead of composed form (LATIN SMALL LETTER A WITH DIAERESIS). You can use iconv to convert text to composed form:

$ echo -n ä | xxd -p
c3a4
$ touch ä
$ ls | tr -d '\n' | xxd -p
61cc88
$ ls | tr -d '\n' | iconv -f utf-8-mac -t utf-8 | xxd -p
c3a4

HFS+ does not use NFD (normal form decomposed). From http://developer.apple.com/library/mac/#qa/qa1173/_index.html:

Important: The terms used in this Q&A, precomposed and decomposed, roughly correspond to Unicode Normal Forms C and D, respectively. However, most volume formats do not follow the exact specification for these normal forms. For example, HFS Plus (Mac OS Extended) uses a variant of Normal Form D in which U+2000 through U+2FFF, U+F900 through U+FAFF, and U+2F800 through U+2FAFF are not decomposed (this avoids problems with round trip conversions from old Mac text encodings).

Something like this might also work:

python -c 'import unicodedata as ud; print ud.normalize("NFC", u"\N{LATIN SMALL LETTER A}\N{COMBINING DIAERESIS}")'

Improve this answer
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .