I simplified my code for better understanding.
here is the problem :
case 1:
# -*- coding: utf-8 -*-
text = "چرا کار نمیکنی؟" # also using u"...." results the same
print(text)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2: character maps to <undefined>
text = "چرا کار نمیکنی؟".encode("utf-8")
print(text)
import sys
text = "چرا کار نمیکنی؟".encode("utf-8")
sys.stdout.buffer.write(text)
چرا کار نمیکنی؟
text = "چرا کار نمیکنی؟" .encode("utf-8")# also using u"...." results the same
print(text)
python persian_encoding.py > test.txt
b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda\xa9\xd9\x86\xdb\x8c\xd8\x9f'
Your code is correct as it works on my computer with both Python 2 and 3 (I'm on OS X):
~$ python -c 'print "تست"'
تست
~$ python3 -c 'print("تست")'
تست
The problem is with your terminal that can not output unicode characters. You could verify it by redirecting your output to a file like python3 my_file.py > test.txt
and open the file using an editor.
If you are on Windows you could use a terminal like Console2 or ConEmu that renders unicode better than Windows prompt.
You may encounter errors with these terminals too because of wrong code-pages/encodings of Windows. There is a small python package that fixes them (sets them correctly):
1- Install this pip install win-unicode-console
2- Put this at the top of your python file:
try:
# Fix UTF8 output issues on Windows console.
# Does nothing if package is not installed
from win_unicode_console import enable
enable()
except ImportError:
pass
If you got errors when redirecting to a file, you may fix it by settings io encoding:
On Windows command line:
SET PYTHONIOENCODING=utf-8
On Linux/OS X terminal:
export PYTHONIOENCODING=utf-8
u"aaa"
syntax in python 3. Strings literals are unicode by default.# -*- coding: utf-8 -*-
) is not needed.