Mic Mic - 1 year ago 138
Python Question

Access code in Firefox Inspector with Python

I'm learning scraping with Python. I use

to extract information from the source code. However, I see that not all the information that are available in
Firefox Inspector
are reported in the source code. Why is it the case?

jDo jDo
Answer Source

The site has a separate JSON API for auction bids.

import requests
import json
from bs4 import BeautifulSoup as soup

data = requests.get("http://www.charitystars.com/ajax/getbidshistory?oid=9315")
data_dict = json.loads(data.text)

auction_table = soup(data_dict["html"])

for td in auction_table.find_all("td"):
    print td.get_text()


€ 120


€ 60


You can simply increment the oid to get all auctions. Be nice to the site though! Add delays and such so you don't end up DOSing a charity!

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download