PythonMastery
reference 4 min read · lesson 37 of 60 in Errors

UnicodeEncodeError: 'ascii' codec can't encode character

1 · The lesson

read

What this error means

You are writing text out (to a file, a terminal, a socket), and the encoding in use cannot represent one of the characters. ASCII can encode 128 characters, and ₹, é and → are not among them.

This is the opposite of UnicodeDecodeError. Decoding turns bytes that come in into text. Encoding turns text that goes out into bytes.

When you see it

python
Traceback (most recent call last):
  File "export.py", line 9, in <module>
    f.write(f"Total: ₹{amount}")
UnicodeEncodeError: 'ascii' codec can't encode character '₹'
in position 7: ordinal not in range(128)

On Windows the codec is usually different, but the cause is the same:

python
UnicodeEncodeError: 'charmap' codec can't encode character '→'
in position 12: character maps to <undefined>

Why it happens

open() uses the platform's preferred encoding when you do not name one. That is UTF-8 on modern Linux and macOS, but in a container with no locale it can be ASCII, and on Windows it has historically been a legacy code page such as cp1252.

So the same code writes a rupee sign without problems on a laptop and fails in the container. This is why the error so often appears only after a deploy.

How to fix it

Name the encoding every time you open a file.

python
with open("report.txt", "w", encoding="utf-8") as f:
    f.write(f"Total: ₹{amount}")
+ setup added so this can run · defines amount
# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

amount = _AutoMock('amount')

There is no good reason to rely on the platform default. Writing encoding="utf-8" is the complete fix, and it makes the code behave the same everywhere.

For stdout in a container, set the environment rather than the code:

dockerfile
ENV PYTHONIOENCODING=utf-8
ENV LANG=C.UTF-8

Python 3.15 makes UTF-8 mode the default, which removes most of these problems. But code that names its encoding is correct on every version.

When the destination really cannot store the character, choose explicitly what to lose:

python
text = "Total: ₹1,200"

text.encode("ascii", errors="ignore")     # b'Total: 1,200'   (drops it silently)
text.encode("ascii", errors="replace")    # b'Total: ?1,200'  (marks the gap with ?)
text.encode("ascii", errors="xmlcharrefreplace")  # b'Total: &#8377;1,200'

Prefer replace over ignore for anything you will read later. A ? tells you something was lost, but ignore produces text that looks correct and is wrong.

For CSV that Excel will open, use utf-8-sig. Excel assumes a legacy code page unless the file begins with a byte-order mark, and this is why exported CSVs so often show ₹ where the rupee sign should be:

python
with open("export.csv", "w", encoding="utf-8-sig", newline="") as f:
    csv.writer(f).writerows(rows)
+ setup added so this can run · defines rows, csv
# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

rows = _AutoMock('rows')
csv = _AutoMock('csv')

When you'd see this in real code

  • A report containing a customer name with an accent, which works locally and fails in the nightly container.
  • print() of a log line with an arrow or emoji, on a Windows console using cp1252.
  • A CSV of Indian prices that opens in Excel as mojibake (garbled characters). This is a related problem: the bytes are fine, but Excel decodes them with the wrong encoding.
  • Any code that writes user-supplied text without naming an encoding. It works until the first user whose name has a non-ASCII character.

See Also

Found this useful? Share it with a friend who’s learning Python.