DJV DJV - 3 months ago 30
Python Question

Python requests - how to set default retries object for created sessions?

I'd like to force all requests to retry on some 5xx HTTP status codes. What I would do is:

retry = requests.packages.urllib3.util.retry.Retry(
total=20,
backoff_factor=0.1,
status_forcelist=[500, 502, 503, 504],
method_whitelist=frozenset(['GET', 'POST']))
for adapter in session.adapters.values():
adapter.max_retries = retry


But I need to do it for existing code with a lot of different sessions in different modules/packages. Some of them use
s = Session(); s.get()
, others use
requests.get()
. So, I'd like to force them all to use this
Retry
instance.

Is it possible to do it on
requests
package level (by initializing, setting up, simple monkey patching
requests
package)?

import requests

# Initialize/setup/patch requests package.
???

# Following should retry on server errors:
requests.Session().get('http://httpstat.us/500')
requests.post('http://httpstat.us/500', data={})


I tried to set
requests.adapters.DEFAULT_RETRIES
to
retry
object. But this is not how it works...

DJV DJV
Answer
import requests
import logging


logging.basicConfig(level='DEBUG')


def forceretry(max_retries):
    """ Decorator for `requests.adapters.HTTPAdapter.__init__`. """
    def decorate(func):
        def wrapper(self, *args, **kwargs):
            func(self, *args, **kwargs)
            self.max_retries = max_retries
        return wrapper
    return decorate


max_retries = requests.packages.urllib3.util.retry.Retry(
        total=20,
        backoff_factor=0.1,
        status_forcelist=[500, 502, 503, 504],
        method_whitelist=frozenset(['GET', 'POST']))

func = requests.adapters.HTTPAdapter.__init__
requests.adapters.HTTPAdapter.__init__ = forceretry(max_retries)(func)

requests.post('http://httpstat.us/500', data={})

It forces Retry object for every Session (doesn't set default one). But it worked for me.