Guide

How to convert a file to UTF-8 on a Mac

Converting to UTF-8 is easy once you know what the file is in now, and that is the hard part: the wrong guess converts without complaint and gives you a file with every accent in it quietly replaced. Here is how to find out, and how to convert without losing anything.

With what the Mac already has

file -I notes.txt guesses the character set in Terminal. It is right about UTF-8 and UTF-16 and says unknown-8bit or iso-8859-1 about almost everything else — which is where the trouble is.

iconv -f WINDOWS-1252 -t UTF-8 notes.txt > notes-utf8.txt converts, if you already know the answer to the first question. Given the wrong source set it converts anyway: Windows-1252 read as Mac Roman turns every ü into a different letter, and nothing tells you.

TextEdit can do it too: open the file, choose Format ▸ Make Plain Text if it is not, and pick an encoding when saving. It shows you the text as it guessed it, which helps, but it will not show you the alternatives.

Why the guess is hard

Every 8-bit character set decodes every byte. A German file written on Windows is valid Mac Roman, valid Latin-1 and valid Windows-1250, so "does it decode?" tells you nothing. What tells them apart is whether the result reads like a language — ü against Ÿ, Czech in Latin-2 against Czech in Latin-1, Russian in KOI8-R against Windows-1251. A short file makes it harder still: a French note with five accented letters is also valid Chinese.

Doing it in Volt

Open the file. The character set is detected — by byte order mark, then structure, then how the text reads, sampled at the start, middle and end — and the status bar names the one it chose. When it was a close call, it says so.

Check it. View ▸ Inspect ▸ Compare Character Sets… shows the same bytes read every plausible way, side by side and scored, and follows your selection so you can point at the word that matters. Only the sets that can genuinely read the bytes are offered.

If the guess was wrong, pick the right set in the status bar and choose to re-read the bytes: nothing is converted, the file is simply shown correctly.

Then convert. Pick UTF-8 and choose to convert; the next save writes UTF-8. Changing your mind is ⌘Z, like any other edit.

Byte order mark or not. File ▸ Byte Order Mark turns it on or off. Volt will not put one into .htaccess or a script with a #! line unless you insist, because those break.

Going the other way

Sometimes a file has to stay in, or go back to, an older set: a system that only reads Windows-1252, a mainframe that wants EBCDIC 273. Volt reads and writes 220 character sets, including 62 EBCDIC code pages, and it tells you before saving if a character in the text has no spelling in the set you chose — nothing becomes a ? without a word.

A whole folder of them

View ▸ Inspect ▸ Project Report… lists every file in a project with its character set, byte order mark and line ending, filtered to Not UTF-8 — so you know exactly which files need converting before you start.

Questions

Related

• Converting line endings on a Mac — CRLF, LF and the rarer ones.
• EBCDIC on macOS — 62 code pages.
• Finding invisible characters — BOMs, zero-width spaces and worse.
• Every file Volt opens — all 220 character sets.