Mr. A Mr. A - 2 years ago 97
Python Question

"Do like readlines()" to a Session object (Python)

I want to pick out a few lines of information from some webpages. I want (or I am) to open the web pages, iterate down through the lines, checking each for a keyword, copying the information I want when I find it.

The pages require a session.

def getpage()
home = 'website'
exstension1 = '/input/page'
extension2 = '/output/page'
indexnumber = '11100'

sess = requests.Session()
getter = sess.get(home+extension1)
payload = {'foo':'bar','indexnumber':indexnumber}
getter =,data=payload)

return sess

As I tried to say in the title, I need a readlines() method for a .get()

a.get(somePage)###Now could I put...###.readlines()
###I don't think I want the following, for performance reasons, correct me if I am wrong
F = open(someNewFile,mode='w')
F = open(thatFileIJustMade).readlines()###All that just to turn it into a File on which I can use readlines?


When I try


I get

AttributeError: Response Object Doesn't have attribute readlines

Answer Source

There are a few ways you can do this, but the most Requests-y way is to use a streaming request along with Response.iter_lines():

r = requests.get(somePage, stream=True)

for line in r.iter_lines(1024):
    # Do stuff on this line.
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download