I have been trying to parse the table contents from here
i have tried a couple of alternatives, like
import requests, lxml.html
url = requests.get(url)
html = lxml.html.fromstring(url.content)
packages = html.xpath('//div[@id="replacetext"]/table/tbody//tr/td/a//text()') # get the text inside all "<tr><td><a ...>text</a></td></tr>"
I tried you code. The problem is not caused by
lxml. It is caused by how you load the webpage.
I know that you use the
requests to get the content of webpage, however, the content you get from
requests may be different from the content you see in the browser.
In this page, 'https://nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm', print the content of
request.get, you will find that the source code of this page contains no table!!! The table is loaded by
So find a way to load the 'really' page you want, the you can use 'lxml`.
By the way, in web scraping, there are also something you need to mention, for example,
request headers. It's a good practice to set your request headers when you do the http request. Some sites may block you, if you do not provide a reasonable
User-Agent in the header. Though there is nothing to do with your current problem.