Take the 2-minute tour ×
Ask Different is a question and answer site for power users of Apple hardware and software. It's 100% free, no registration required.

I'm working with UTF-8 files, but TextEdit seems to not fully support the BOM. When a BOM is present, it DOES render unicode characters correctly. However, upon saving the file, the BOM is removed, so subsequently opening the file in TextEdit causes 'the wrong' characters to be displayed.

Is there a way of remedying this? Bear in mind I'm sharing files with several non-technical users, so anything related to the command-line is out of the question. If the solution is 'use another text editor', this might just about be acceptable, if it's free, and not totally different from TextEdit - i.e. the simpler, the better. But, ideally, there's a way of fixing TextEdit ...

share|improve this question
2  
Why use the BOM? Simply using UTF-8 without BOM avoids so many issues. –  Mathias Bynens Mar 29 '12 at 10:31
    
Because TextEdit doesn't recognise the file as UTF-8 without it. Or, possibly, it's an artifact of emailing the file as an attachment - either way, adding the BOM resolves the problem. –  Bobby Jack Mar 29 '12 at 10:36
    
TextEdit automatically figures out the file encoding the open dialogue. If it can't, such as your case, then you can choose UTF-8 from the pulldown menu. If you save a copy of the file then you get the choice of choosing the encoding from the pulldown menu as well. –  ismail Mar 29 '12 at 11:03
    
@BobbyJack Check out my answer, I’m sure it fixes your problem. –  Mathias Bynens Mar 29 '12 at 16:30
add comment

3 Answers

up vote 3 down vote accepted

TextEdit → Preferences → Open and Save

Screenshot

Set “Plain Text File Encoding” to “Unicode (UTF-8)” for both opening and saving files.

If that doesn’t fix it already, open Terminal.app and enter the following:

echo "0x08000100:0" > ~/.CFUserTextEncoding

Then logout and login again.

This fixes the ancient UTF-8 bug in OS X that also manifest in QuickLook.

To fix it for just a single file rather than for all files (I don’t see why you would, though), just write the extended attribute that TextEdit uses:

xattr -w com.apple.TextEncoding 'UTF-8;134217984' filename.txt

Hope this helps!

share|improve this answer
    
Can you please shed some light on what the string for the ~/. CFUserTextEncoding means and represents? –  ismail Mar 29 '12 at 23:52
1  
@ismail It’s all explained in the blog post that I linked to, but here’s a quick summary: it sets the default encoding to UTF-8 (first number 0x08000100) and the language to English (second number 0). The default content is 0:0 (MacRoman and English). Here’s some more info on the ~/.CFUserTextEncoding file. –  Mathias Bynens Mar 30 '12 at 5:30
    
Thanks. I set the CFUserTextEncoding and the corresponding ENV. Didn't notice anything different so far, though I deal with lot's of other encodings. Hopefully it would help in the future. –  ismail Mar 30 '12 at 6:52
    
@ismail Don’t forget to logout and back in again! After that, your problem should be fixed. –  Mathias Bynens Mar 30 '12 at 6:57
add comment

I think using TextWrangler to always save as "UTF-8 with BOM" is the best way to make sure all other systems will see your files as UTF-8. Another option would be to use UTF-16 instead, but that is unusual and could confuse some users.

share|improve this answer
add comment

This isn't the main solution you're looking for, but TextWrangler is free, handles UTF-8 (and many other encodings) very well, and doesn't have too steep a learning curve, particularly if you're just using functionality similar to TextEdit. I know you'd like to fix TextEdit, and I hope you get an answer explaining how to do that, but in the absence of a primary solution, this should be a good backup plan.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.