delavnog delavnog - 1 year ago 101
Python Question

Log all requests from the python-requests module

I am using python Requests. I need to debug some

activity, and for that I would like it to log all requests being performed. I could get this information with
, but unfortunately it is not possible to grep https connections (which are needed for

How can I activate logging of all URLs (+ parameters) that
is accessing?

Answer Source

The underlying urllib3 library logs all new connections and URLs with the logging module, but not POST bodies. For GET requests this should be enough:

import logging


which gives you the most verbose logging option; see the logging HOWTO for more details on how to configure logging levels and destinations.

Short demo:

>>> import requests
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> r = requests.get('')
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1):
DEBUG:requests.packages.urllib3.connectionpool:"GET /get?foo=bar&baz=python HTTP/1.1" 200 353

The following messages are logged:

  • INFO: New connections (HTTP or HTTPS)
  • INFO: Dropped connections
  • INFO: Redirects
  • WARN: Connection pool full (if this happens often increase the connection pool size)
  • WARN: Retrying the connection
  • DEBUG: Connection details: method, path, HTTP version, status code and response length