Damodaran Damodaran - 19 days ago 8
Python Question

How to set and retrieve cookie in HTTP header in Python?

I need to get the cookies from a HTTP response sent by a server and put it in the next request's header. How can I do it?

Thanks in advance.

Answer

You should use the cookielib module with urllib.

It will store cookies between requests, and you can load/save them on disk. Here is an example:

import cookielib
import urllib2

cookies = cookielib.LWPCookieJar()
handlers = [
    urllib2.HTTPHandler(),
    urllib2.HTTPSHandler(),
    urllib2.HTTPCookieProcessor(cookies)
    ]
opener = urllib2.build_opener(*handlers)

def fetch(uri):
    req = urllib2.Request(uri)
    return opener.open(req)

def dump():
    for cookie in cookies:
        print cookie.name, cookie.value

uri = 'http://www.google.com/'
res = fetch(uri)
dump()

res = fetch(uri)
dump()

# save cookies to disk. you can load them with cookies.load() as well.
cookies.save('mycookies.txt')

Notice that the values for NID and PREF are the same between requests. If you omitted the HTTPCookieProcessor these would be different (urllib2 wouldn't send Cookie headers on the 2nd request).