maxwell maxwell - 2 months ago 32
Python Question

How to clear cookies in scrapy?

By default, scrapy stores and passes cookies along requests. But how do I access or clear the stored cookies at certain point in the spider? Thanks?

Answer Source

to set cookies to a specific request use request cookies field for example from docs:

request_with_cookies = Request(url="http://www.example.com",
                               cookies={'currency': 'USD', 'country': 'UY'})

do access request cookies:

request.headers.getlist('Cookie')

response cookies:

response.headers.getlist('Set-Cookie')

for more details see cookies middleware