lakshman lakshman - 3 months ago 20
Java Question

403 error while getting the google result using jsoup

I'm trying to get Google results using the following code:

Document doc = con.connect("http://www.google.com/search?q=lakshman").timeout(5000).get();


But I get this exception:

org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403,URL=http://www.google.com/search?q=lakshman


A 403 error means the server is forbidding access, but I can load this URL in a web browser just fine. Why does Jsoup get a 403 error?

Answer

You just need to add the UserAgent property to HTTP header as follows:

Jsoup.connect(itemUrl)
     .userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36")
     .get()