I have this website and I want to download the content of the page.
I tried selenium, and button clicking with it, but with no success.
from contextlib import closing
from selenium.webdriver import Firefox
with closing(Firefox()) as browser:
# setting the url
# finding and clicking the button
button = browser.find_element_by_id('select_button')
page = browser.page_source
I always try to avoid selenium like the plague when scraping; it's very slow and is almost never the best way to go about things. You should dig into the source more before scraping; it was clear on this page that the html was coming in and then a separate call was being made to get the table's data. Why not make the same call as the page? It's lightning fast and requires no html parsing; just returns raw data, which seems to be what you're looking for. the python requests import is perfect for this. Happy Scraping!
import requests table_data = requests.get('http://bonusbagging.co.uk/odds-server/getdata_slow.php').content