3

I'm working with UTF-8 files, but TextEdit seems to not fully support the BOM. When a BOM is present, it DOES render unicode characters correctly. However, upon saving the file, the BOM is removed, so subsequently opening the file in TextEdit causes 'the wrong' characters to be displayed.

Is there a way of remedying this? Bear in mind I'm sharing files with several non-technical users, so anything related to the command-line is out of the question. If the solution is 'use another text editor', this might just about be acceptable, if it's free, and not totally different from TextEdit - i.e. the simpler, the better. But, ideally, there's a way of fixing TextEdit ...

Improve this question
5
  • 3
    Why use the BOM? Simply using UTF-8 without BOM avoids so many issues.
    – Mathias Bynens
    Commented Mar 29, 2012 at 10:31
  • Because TextEdit doesn't recognise the file as UTF-8 without it. Or, possibly, it's an artifact of emailing the file as an attachment - either way, adding the BOM resolves the problem.
    – Bobby Jack
    Commented Mar 29, 2012 at 10:36
  • 1
    TextEdit automatically figures out the file encoding the open dialogue. If it can't, such as your case, then you can choose UTF-8 from the pulldown menu. If you save a copy of the file then you get the choice of choosing the encoding from the pulldown menu as well.
    – ismail
    Commented Mar 29, 2012 at 11:03
  • @BobbyJack Check out my answer, I’m sure it fixes your problem.
    – Mathias Bynens
    Commented Mar 29, 2012 at 16:30
  • @MathiasBynens There are some stubborn programs that refuse to recognize UTF8 characters without BOM marker. One example is Excel - it will screw up Latvian characters in csv files even when the file is saved as proper UTF8. Only UTF8 with BOM makes Excel happy.
    – JustAMartin
    Commented Aug 17, 2022 at 7:35

3 Answers 3

Reset to default
5

TextEdit → Preferences → Open and Save

Screenshot

Set “Plain Text File Encoding” to “Unicode (UTF-8)” for both opening and saving files.

If that doesn’t fix it already, open Terminal.app and enter the following:

echo "0x08000100:0" > ~/.CFUserTextEncoding

Then logout and login again.

This fixes the ancient UTF-8 bug in OS X that also manifest in QuickLook.

To fix it for just a single file rather than for all files (I don’t see why you would, though), just write the extended attribute that TextEdit uses:

xattr -w com.apple.TextEncoding 'UTF-8;134217984' filename.txt

Hope this helps!

Improve this answer
4
  • Can you please shed some light on what the string for the ~/. CFUserTextEncoding means and represents?
    – ismail
    Commented Mar 29, 2012 at 23:52
  • 1
    @ismail It’s all explained in the blog post that I linked to, but here’s a quick summary: it sets the default encoding to UTF-8 (first number 0x08000100) and the language to English (second number 0). The default content is 0:0 (MacRoman and English). Here’s some more info on the ~/.CFUserTextEncoding file.
    – Mathias Bynens
    Commented Mar 30, 2012 at 5:30
  • Thanks. I set the CFUserTextEncoding and the corresponding ENV. Didn't notice anything different so far, though I deal with lot's of other encodings. Hopefully it would help in the future.
    – ismail
    Commented Mar 30, 2012 at 6:52
  • @ismail Don’t forget to logout and back in again! After that, your problem should be fixed.
    – Mathias Bynens
    Commented Mar 30, 2012 at 6:57
5

This isn't the main solution you're looking for, but TextWrangler is free, handles UTF-8 (and many other encodings) very well, and doesn't have too steep a learning curve, particularly if you're just using functionality similar to TextEdit. I know you'd like to fix TextEdit, and I hope you get an answer explaining how to do that, but in the absence of a primary solution, this should be a good backup plan.

Improve this answer
1

I think using TextWrangler to always save as "UTF-8 with BOM" is the best way to make sure all other systems will see your files as UTF-8. Another option would be to use UTF-16 instead, but that is unusual and could confuse some users.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .