Soorena Soorena - 1 year ago 141
Python Question

Python 3 print() function with Farsi/Arabic characters

I simplified my code for better understanding.
here is the problem :

case 1:

# -*- coding: utf-8 -*-

text = "چرا کار نمیکنی؟" # also using u"...." results the same


UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2: character maps to <undefined>

case 2:

text = "چرا کار نمیکنی؟".encode("utf-8")

there is no output.

case 3:

import sys

text = "چرا کار نمیکنی؟".encode("utf-8")


چرا کار نمیکنی؟

I know that case 3 works somehow , but I want to use other functions like print() , write(str()) , ....

I also read the documentation of python 3 regarding to Unicode here.

and also read dozens of Q&A in stackoverflow.

and here is a long article explaining the problem and answer for python 2.X

the simple question is:

how to print non-ASCII characters like Farsi or Arabic using python print() function?

update 1 :
as it is suggested from many guys that the problem is concerned with the terminal I tested the case :

case 4 :

text = "چرا کار نمیکنی؟" .encode("utf-8")# also using u"...." results the same

terminal :

python > test.txt

test.txt :

b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda\xa9\xd9\x86\xdb\x8c\xd8\x9f'

Answer Source

Your code is correct as it works on my computer with both Python 2 and 3 (I'm on OS X):

~$ python -c 'print "تست"'
~$ python3 -c 'print("تست")'

The problem is with your terminal that can not output unicode characters. You could verify it by redirecting your output to a file like python3 > test.txt and open the file using an editor.

If you are on Windows you could use a terminal like Console2 or ConEmu that renders unicode better than Windows prompt.

You may encounter errors with these terminals too because of wrong code-pages/encodings of Windows. There is a small python package that fixes them (sets them correctly):

1- Install this pip install win-unicode-console

2- Put this at the top of your python file:

    # Fix UTF8 output issues on Windows console.
    # Does nothing if package is not installed
    from win_unicode_console import enable
except ImportError:

If you got errors when redirecting to a file, you may fix it by settings io encoding:

On Windows command line:


On Linux/OS X terminal:


Some points

  • There is no need to use u"aaa" syntax in python 3. Strings literals are unicode by default.
  • Default coding of files is UTF8 in python 3 so coding declaration comment (e.g. # -*- coding: utf-8 -*-) is not needed.
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download