Leva7 Leva7 - 11 months ago 115
Python Question

How to change the form encoding in Python Requests?

I have a Cyrillic string that I'd like to send as form data in Windows-1251 encoding using Python Requests.

Take a look at this example:

import requests
st = 'Искать'
requests.post('http://localhost:8888', data={'test': st})


However, the data of the request becomes this (running
nc -l 8888
to view the request):

test=%D0%98%D1%81%D0%BA%D0%B0%D1%82%D1%8C


That is my target string, encoded as UTF-8, which we can easily check:

>>> from urllib.parse import quote
>>> st = 'Искать'
>>> quote(st, encoding='utf-8') # Matches
'%D0%98%D1%81%D0%BA%D0%B0%D1%82%D1%8C'
>>> quote(st, encoding='cp1251') # Doesn't match
'%C8%F1%EA%E0%F2%FC'


Now I thought that if I encode the data myself, it would work just fine.

requests.post('http://localhost:8888', data={'test': quote(st, encoding='cp1251')})


But turns out that this is not the case because while the letters are encoded, the percent signs get encoded again into UTF-8 (
%25
) and the whole string becomes invalid again

test=%25C8%25F1%25EA%25E0%25F2%25FC


So I'm looking for a way to either disable the built-in Requests' encoding feature or override the encoding value. How can I do it?

I'm using Python 3.5, Requests 2.18.4

Answer Source

From docs.python-requests:

There are times that you may want to send data that is not form-encoded. If you pass in a string instead of a dict, that data will be posted directly.

So, if you don't want your data to get form-encoded you should use a string.

st = 'Искать'
data={'test': quote(st, encoding='cp1251')}
data = '&'.join('='.join(i) for i in data.items())
requests.post('http://localhost:8888', data=data)
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download