Thomas Thomas - 7 months ago 104
Python Question

Python 3 - POST request with urllib.request returning HTML data

I'm trying to access a website's API with a POST request but it's returning HTML and I don't know why.

import urllib.request
url = 'https://matchbook.com'
loginReq = '"method": "/bpapi/rest/security/session", "params":{"username": "xxxxx","password": "xxxxx"}'

req = urllib.request.Request(url, loginReq.encode('utf-8'))
response = urllib.request.urlopen(req).read()
print (response)


This returns a stream of HTML which basically says I'm using an unsupported version of Internet Explorer and I should consider upgrading to a newer one.

I've spent some time looking online and it's hard to find any detailed information about get/post/put/delete requests in Python 3, most of the helpful information I've found involves using the "requests" module in Python 2. Would switching to Python 2 make this easier?

Here's the information they give in the documentation:

Resource URL:
URL: matchbook.com URN: /bpapi/rest/security/session

Example request:
POST /security/session
{
"username": "j_henry",
"password": "**"
}

Example response:
{
"session-token": "1418_1234567890",
"user-id": 1418,
"account": { // Same as GET /account API response.
...
}

Answer

The problem is almost certainly that you're incorrectly sending data to the API; aka your loginReq variable.

You need to use curl or some equivalent tool to figure out exactly what parameters and values need to be posted. Something along the lines of

curl --data "method:/bpapi/rest/security/session&params={username=j_henry&password=**}" https://www.matchbook.com

Once you have the request down you can start worrying about actually programming it.