bethu bethu - 1 year ago 49
Python Question

HTML data grabbing in python?

I'm fairly new to programming and I am trying to take data from a webpage and use it in my python code. Basically, I'm trying to take the price of an item for a game by having python grab the data whenever I run my code, if that makes sense. Here's what I'm struggling with in particular:

The HTML page I'm using is for runescape, namely

This page provides me with a bunch of dictionaries from which I am trying to extract the price of the item in question. All I really want to do it get all of this data into python so I can then manipulate it. My current code is:

import urllib2

response =urllib2.urlopen('')

print response

And it outputs:

addinfourl at 49631760 whose fp = socket._fileobject object at 0x02F4B2F0

whereas I just want it to display exactly what is on the URL in question.

Any ideas? I'm sorry if my formatting is terrible. And if it sounds like I have no idea what I'm talking about, it's because I don't.

Answer Source

If the webpage returns a json-encoded data, then do something like this:

import urllib2
import json

response = urllib2.urlopen("")
data = json.load(response)   


Extract the relevant keys in the data variable to get the values you want.