Helen Neely Helen Neely - 1 month ago 11
Python Question

How can I read the contents of an URL with Python?

The following works when I paste it on the browser:

http://www.somesite.com/details.pl?urn=2344


But when I try reading the URL with Python nothing happens:

link = 'http://www.somesite.com/details.pl?urn=2344'
f = urllib.urlopen(link)
myfile = f.readline()
print myfile


Do I need to encode the URL, or is there something I'm not seeing?

Answer Source

To answer your question:

import urllib

link = "http://www.somesite.com/details.pl?urn=2344"
f = urllib.urlopen(link)
myfile = f.read()
print myfile

You need to read(), not readline()

Or, just get this library here: http://docs.python-requests.org/en/latest/ and seriously use it :)

import requests

link = "http://www.somesite.com/details.pl?urn=2344"
f = requests.get(link)

print f.text