Wouter Wouter - 4 months ago 12
Linux Question

Python3 UnicodeEncodeError when run via Synology task scheduler

I get a Python3 UnicodeEncodeError when I run my script via the Synology task scheduler. I do not get this error when I run the script via the commandline (using PuTTY). Why is this and how can I solve it?

Simple test script:

import sys
print (sys.version) # to confirm the correct Python version
print("Fichier non trouvé♠ #M–Nein") # to test non ascii characters
test = "Fichier non trouvé♠ #M–Nein"
print ("test is " + test)
test2 = str(test) # to test if the string function causes and issue
print ("test2 is " + test2)


Commandline output:

admin@DiskStation:/volume1/@appstore/py3k/usr/local/bin$ /volume1/@appstore/py3k/usr/local/bin/python3 /volume1/Documenten/MyPythonScripts/Test.py
3.5.1 (default, Feb 23 2016, 17:46:04)
[GCC 4.9.3 20150311 (prerelease)]
Fichier non trouvé♠ #M–Nein
test is Fichier non trouvé♠ #M–Nein
test2 is Fichier non trouvé♠ #M–Nein


Task scheduler output:

3.5.1 (default, Feb 23 2016, 17:46:04)
[GCC 4.9.3 20150311 (prerelease)]
Traceback (most recent call last):
File "/volume1/Documenten/MyPythonScripts/Test.py", line 3, in <module>
print("Fichier non trouv\xe9\u2660 #M\u2013Nein") # to test non ascii characters
UnicodeEncodeError: 'ascii' codec can't encode characters in position 17-18: ordinal not in range(128)


Note: The same Python version and script are run using

/volume1/@appstore/py3k/usr/local/bin/python3
/volume1/Documenten/MyPythonScripts/Test.py


in both situations.

Note2: An earlier (line 1) Unicode error occurs if I run the script via the commandline but using Python2.7: (FYI below, Python 3 vs Python 2)

admin@DiskStation:/volume1/Documenten/MyPythonScripts$ **python3** Test.py
Fichier non trouvé♠ #M–Nein
test is Fichier non trouvé♠ #M–Nein
test2 is Fichier non trouvé♠ #M–Nein
admin@DiskStation:/volume1/Documenten/MyPythonScripts$ **python** Test.py
File "Test.py", line 1
SyntaxError: Non-ASCII character '\xc3' in file Test.py on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details


This Unicode issue can be solved in Python2.7 by adding the following as 1st or 2nd line to the script:

# -*- coding: UTF-8 -*-


Then the script runs fine from the command line.

But adding this UTF-8 line does not resolve the issue with running the script from the Synology Task Scheduler, then the error is still raised?!:

3.5.1 (default, Feb 23 2016, 17:46:04)
[GCC 4.9.3 20150311 (prerelease)]
Traceback (most recent call last):
File "/volume1/Documenten/MyPythonScripts/Test.py", line 4, in <module>
print("Fichier non trouv\xe9\u2660 #M\u2013Nein") # to test non ascii characters
UnicodeEncodeError: 'ascii' codec can't encode characters in position 17-18: ordinal not in range(128)

Answer

When running from the command line, Python detects the encoding of the terminal and encodes Unicode text to the terminal in that encoding. When running under your task scheduler, Python is not detecting the output encoding and is defaulting to ascii.

It works in Python 2 when declaring the source encoding as utf8, because you are not using Unicode strings and print just sends the UTF-8-encoded byte string to the terminal. Your terminal is UTF-8, so it works.

You can override Python's default assumptions by setting the environment variable PYTHONIOENCODING=utf8 when running under the scheduler. This variable is available under all platforms.

Ref: PYTHONIOENCODING

Comments