Julek Julek - 24 days ago 8
Javascript Question

Node.js equivalent of this curl request

I'm trying to use the HTML validator API. The curl examples work fine for me and I can run them find in Node as a child process. Here is the code for that:

var command = ('curl -H "Content-Type:text/html; charset=utf-8" --data-binary @' + file +
' https://validator.w3.org/nu/?out=json');

exec(command, function(err1, out, err2) {
console.log(out);
console.log('done');
});


However, when I tried to use a standard HTTP request, I couldn't get it working. I tried it with the Unirest library for Node. Here is the code I used:

var source = '<html><head><title>a</title></head><body>a</body></html>';
var url = 'http://validator.w3.org/nu/?out=json';


var Request = unirest.post(url);
Request.headers({'Content-Type': 'text/html', 'charset': 'utf-8'});
Request.send(source);
Request.end(res => console.log(res));


The response body is undefined and the response raw_body is empty. I don't know what I'm doing wrong and would appreciate any help.

Answer

Seems validator.w3.org won't respond to requests without a user-agent header. Add this header:

Request.headers({'Content-Type': 'text/html; charset=utf-8', 'user-agent': 'Node.js'});

Or use whatever useragent you want.