Luis Cruz Luis Cruz - 2 months ago 18
Python Question

Scraping a specific table in selenium

I am trying to scrape a table found inside a div on a page.

Basically here's my attempt so far:

# NOTE: Download the chromedriver driver
# Then move exe file on C:\Python27\Scripts
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import sys

driver = webdriver.Chrome()
driver.implicitly_wait(10)

URL_start = "http://www.google.us/trends/explore?"
date = '&date=today%203-m' # Last 90 days
location = "&geo=US"
symbol = sys.argv[1]
query = 'q='+symbol
URL = URL_start+query+date+location

driver.get(URL)

table = driver.find_element_by_xpath('//div[@class="line-chart"]/table/tbody')

print table.text


If I run the script, with an argument like "stackoverflow" I should be able to scrape this site: https://www.google.us/trends/explore?date=today%203-m&geo=US&q=stackoverflow

Apparently the xpath I have there is not working, the program is not printing anything, it's just plain blank.

I am basically in need on the values of the chart that appears on that website. And those values (and dates) are inside a table, here is a screenshot:

enter image description here

Could you help me locate the correct xpath of the table to retrieve those values using selenium on python?

Thanks in advance!

Answer

you can use Xpath As Follow:

//div[@class="line-chart"]/div/div[1]/div/div/table/tbody/tr