I have been using Amazon's Product Advertising API to generate urls that contains prices for a given book. One url that I have generated is the following:
When I click on the link or paste the link on the address bar, the web page loads fine. However, when I execute the following code I get an error:
url = "http://rads.stackoverflow.com/amzn/click/0415376327"
html_contents = urllib2.urlopen(url)
html_contents = urllib2.urlopen("http://rads.stackoverflow.com/amzn/click/0415376327")
It's because Amazon don't allow automated access to their data, so they're rejecting your request because it didn't come from a proper browser. If you look at the content of the 503 response, it says:
To discuss automated access to Amazon data please contact firstname.lastname@example.org. For information about migrating to our APIs refer to our Marketplace APIs at https://developer.amazonservices.com/ref=rm_5_sv, or our Product Advertising API at https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html/ref=rm_5_ac for advertising use cases.
This is because the
User-Agent for Python's
urllib is so obviously not a browser. You could always fake the
User-Agent, but that's not really good (or moral) practice.
As a side note, as mentioned in another answer, the
requests library is really good for HTTP access in Python.