Candroid Candroid - 6 months ago 19
Node.js Question

Request a GET failed in Node

I am requesting a

GET
to a 3rd party api service from my
node
back-end.

I am getting a response of
403 forbidden
:

request("http://www.giantbomb.com/api/search/?api_key=my_api_key&field_list=name,image,id&format=json&limit=1&query=street%20fighter%203&resources=game",(err,res,body) => {
console.log(body);
})


Querying the same request in my browser return the expected results.

Any idea why this can happen?

EDIT:

Logging the response body, I receive the following page (without the JS):

<h1>Wordpress RSS Reader, Anonymous Bot or Scraper Blocked</h1>
<p>
Sorry we do not allow WordPress plugins to scrape our site. They tend to be used maliciously to steal our content. We do not allow scraping of any kind.
You can load our RSS feeds using any other reader but you may not download our content.
<a href='/feeds'>Click here more information on our feeds</a>
</p>
<p>
Or you're running a bot that does not provide a unique user agent.
Please provide a UNIQUE user agent that describes you. Do not use a default user agent like "PHP", "Java", "Ruby", "wget", "curl" etc.
You MUST provide a UNIQUE user agent. ...and for God's sake don't impersonate another bot like Google Bot that will for sure
get you permanently banned.
</p>
<p>
Or.... maybe you're running an LG Podcast player written by a 10 year old. Either way, Please stop doing that.
</p>

Answer

this service requires User-Agent in headers, see this example

const rp = require('request-promise')

const options = {
  method: 'GET',
  uri: 'http://www.giantbomb.com/api/search/?api_key=my_api_key&field_list=name,image,id&format=json&limit=1&query=street%20fighter%203&resources=game',
  headers: { 'User-Agent': 'test' },
  json: true
}

rp(options)
  .then(result => {
    // process result
  })
  .catch(e => {
    // handle error
  })
Comments