mike11 mike11 - 7 days ago 6
JSON Question

Error with jsonlite package in R

Has anyone ever received this error when trying to web scrape a site:

Error in feed_push_parser(readBin(con, raw(), n), reset = TRUE) :
lexical error: invalid char in json text.
<!doctype html><html xmlns="htt
(right here) ------^


I do not understand why I am receiving this error when I scraped the first page of the site no problem with the same declaration on that first page. But the second page I get this error. Is there a way around this?

This works fine:

jsonlite::fromJSON("https://www.reddit.com/r/BestOfStreamingVideo/.json", flatten = TRUE)


Get the error here:

jsonlite::fromJSON("https://www.reddit.com/r/BestOfStreamingVideo/?count=25&after=t3_5fvgls/.json", flatten = TRUE)

Answer

The latter one does not return a JSON. It returns HTML. Enter both URL's in the browser and you'll see the difference.

I guess the URL you are looking for is:

https://www.reddit.com/r/BestOfStreamingVideo/.json?count=25&after=t3_5fvgls/

You need to put the ./json first and append the URL parameter after.