Backus Backus - 5 months ago 31
Python Question

Overriding urllib2.HTTPError or urllib.error.HTTPError and reading response HTML anyway

I receive a 'HTTP Error 500: Internal Server Error' response, but I still want to read the data inside the error HTML.

With Python 2.6, I normally fetch a page using:

import urllib2
url = ""
data = urllib2.urlopen(url)
data =

When attempting to use this on the failing URL, I get the exception

urllib2.HTTPError: HTTP Error 500: Internal Server Error

How can I fetch such error pages (with or without
), all while they are returning Internal Server Errors?

Note that with Python 3, the corresponding exception is


The HTTPError is a file-like object. You can catch it and then read its contents.

    resp = urllib2.urlopen(url)
    contents =
except urllib2.HTTPError, error:
    contents =