Can we use Scrapy for getting content from a web page which is loaded by java script ?
I'm trying to scrap usage examples from this page
but since they are loaded using java script as a Jason object I'm not able to get them Scrapy.
Could u suggest what is the best way to deal with such issues?
Open your browser's developer tools and look at the Network tab. If you hit the "next" button on that page enough, it'll send out a new request:
After removing the JSONP paramter, the URL is pretty straightforward:
By making the minimal number of requests, your spider will be fast.