mirth23 mirth23 - 1 month ago 21
Python Question

python 3 smtplib: binary attachment encodes incorrectly in flask when python-gnupg is active

I am trying to send a binary attachment using the python 3.5.2 smtplib over TLS. My platform is OSX and I am using python installed from homebrew.

When I receive the attachment, the encoding appears to be munged. Instead of the original file that starts with this hex:

ffd8 ffe0 0010 4a46 4946 0001 0100 0001

i.e., <FF><D8><FF><E0>^@^PJFIF^@^A^A


the starting hex of my attachment as received has some weird base64 leftovers:

5c75 6463 6666 5c75 6463 6438 5c75 6463 6666 5c75 6463 6530 0010 4a46 4946 0001

i.e., \udcff\udcd8\udcff\udce0^@^PJFIF


This is a minimal case that fails, which is pretty much exactly what is in the official documentation with the exception of the addition of the TLS logic and gnupg:

import gnupg
gpg = gnupg.GPG('/path/to/gpg')

import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.image import MIMEImage

def send_my_email():
msg = MIMEMultipart()
msg['Subject'] = 'subject'
msg['From'] = 'XXXX@gmail.com'
msg['To'] = 'YYYY@gmail.com'
with open('/tmp/image.jpg', mode='rb') as image_file:
image = MIMEImage(image_file.read())
msg.attach(image)
s = smtplib.SMTP('smtp.gmail.com', 587)
s.starttls()
s.login('XXXX@gmail.com', 'password')
s.send_message(msg)
s.quit()


Based on another question on here, I tried this instead of
send_message()
and it also failed:

s.sendmail('XXXX@gmail.com', ['YYYY@gmail.com'], msg.as_string())


I have also tried explicitly adding
_subtype='jpg'
when I init
MIMEImage
, adding a
Content-Transfer-Encoding
header, and adding a
Content-Disposition
header and none of those seemed to make a difference.

I have verified that I do not have a problem with my email client when it receives base64 encoded attachments from other clients.

I looked at the smtplib source and noticed that the way smtplib handles line separators looks a little odd and wonder if this is possibly related. (also see ref: https://bugs.python.org/issue14645)

Do I need to encode something differently, set something special for my platform, or is this a glitch? Thanks!




Update: This problem only exists when I am running Flask and does not occur outside of Flask. I am in the process of attempting to isolate the problem inside my Flask environment. I thought it might be flask_mail but removing that did not solve the issue. The below code fails when run from Flask on my system, but not if I run it from a shell script from within the same virtual environment and same python binary. I don't expect any answers at this point due to complexity but will leave this open for posterity.

Update 2: I have narrowed this problem down to an interaction with the python-gnupg library from https://github.com/isislovecruft/python-gnupg/. I have updated my minimal example to reflect this. This monkey-patch of python codecs in python-gnupug is responsible for the problem:
codecs.register_error('strict', codecs.replace_errors)

Answer

This was caused due to a bad interaction between python-gnupg and smtplib + MIME. python-gnupg calls codecs.register_error('strict', codecs.replace_errors), which interferes with encoding carried out by other packages.

ref: https://github.com/isislovecruft/python-gnupg/issues/49

Comments