Take the 2-minute tour ×
Ask Different is a question and answer site for power users of Apple hardware and software. It's 100% free, no registration required.

I have two issues that seem related. When copying text or filenames containing umlauts or e.g. å ä ö, OS X can't seem to handle the characters in a sane way. Apparantly this is a well-know problem, a few examples:

http://support.wandisco.com/index.php?/Knowledgebase/Article/View/401/55/problems-with-umlauts--in-filenames-on-mac-os-x http://tex.stackexchange.com/questions/94418/os-x-umlauts-in-utf8-nfd-yield-package-inputenc-error-unicode-char-u8-not

Here is an example of the usual result when copying text from a filename or from inside a pdf in Preview and pasting it in an editor. The first line is the result, the other one is corrected.

example of copy

The difference is clearly visible, since the current font (Courier Prime) doesn't support the first version

Is there any way to fix this? Alternatively, is there any OS X-service available to "clean up" text or normalize it in the correct way?

share|improve this question
1  
I don't understand what normalization has to do with your problem, which looks like a font substitution issue. –  Tom Gewecke Mar 1 '13 at 17:25
    
PS The reason for font substitution may be that Courier Prime is a somewhat limited font and lacks combining diacritical marks. The copy I downloaded does not seem to have any. That would mean it cannot handle any decomposed accented characters and normalization form C is required. I think the best fix may be to use a more complete Unicode font instead. They all normally include combining diacritics as far as I know. –  Tom Gewecke Mar 1 '13 at 19:42
add comment

1 Answer

up vote 0 down vote accepted

HFS+ requires filenames to be in decomposed form (LATIN SMALL LETTER A + COMBINING DIAERESIS) instead of composed form (LATIN SMALL LETTER A WITH DIAERESIS). You can use iconv to convert text to composed form:

$ echo -n ä | xxd -p
c3a4
$ touch ä
$ ls | tr -d '\n' | xxd -p
61cc88
$ ls | tr -d '\n' | iconv -f utf-8-mac -t utf-8 | xxd -p
c3a4

HFS+ does not use NFD (normal form decomposed). From http://developer.apple.com/library/mac/#qa/qa1173/_index.html:

Important: The terms used in this Q&A, precomposed and decomposed, roughly correspond to Unicode Normal Forms C and D, respectively. However, most volume formats do not follow the exact specification for these normal forms. For example, HFS Plus (Mac OS Extended) uses a variant of Normal Form D in which U+2000 through U+2FFF, U+F900 through U+FAFF, and U+2F800 through U+2FAFF are not decomposed (this avoids problems with round trip conversions from old Mac text encodings).

Something like this might also work:

python -c 'import unicodedata as ud; print ud.normalize("NFC", u"\N{LATIN SMALL LETTER A}\N{COMBINING DIAERESIS}")'

share|improve this answer
    
This did help, thanks! –  trmdttr Mar 4 '13 at 8:34
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.