Taku Taku -4 years ago 179
Python Question

Python 'requests' library - define specific DNS?

In my project I'm handling all HTTP requests with python


Now, I need to query the http server using specific DNS - there are two environments, each using its own DNS, and changes are made independently.

So, when the code is running, it should use DNS specific to the environment, and not the DNS specified in my internet connection.

Has anyone tried this using python-requests? I've only found solution for urllib2:


Answer Source

requests uses urllib3, which ultimately uses httplib.HTTPConnection as well, so the techniques from https://stackoverflow.com/questions/4623090/python-set-custom-dns-server-for-urllib-requests still apply, to a certain extent.

The requests.packages.urllib3.connection module subclasses httplib.HTTPConnection under the same name, having replaced the .connect() method with one that calls self._new_conn. It is perhaps easiest to patch that method:

from requests.packages.urllib3.connection import HTTPConnection

def patched_new_conn(self):
    """ Establish a socket connection and set nodelay settings on it

    :return: a new socket connection
    # resolve hostname to an ip address; use your own
    # resolver here, as otherwise the system resolver will be used.
    hostname = your_dns_resolver(self.host)
        conn = socket.create_connection(
            (hostname, self.port),
    except AttributeError: # Python 2.6
        conn = socket.create_connection(
            (hostname, self.port),
    conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY,
    return conn

HTTPConnection._new_conn = patched_new_conn

and you'd provide your own code to resolve self.host into an ip address instead of relying on the socket.create_connection() call resolving the hostname for you.

Like all monkeypatching, be careful that the code hasn't significantly changed in later releases; the patch here was created against requests 2.3.0.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download