Rob Lourens Rob Lourens -3 years ago 195
Node.js Question

Azure Function - port exhaustion in node app

I'm trying to build an Azure Function that makes a large number of outgoing HTTP requests. Periodically it seems to hit a limit and all requests time out for a couple minutes. Then it starts working again and requests go through.

From searching around, it sounds like this might be due to port exhaustion on the machine running the function, but I don't understand how to debug this or work around it in a Node app using the

request
library. It also sounds like Node is supposed to pool connections to prevent this. I'm not at all sure port exhaustion is the problem, since I can't use
netstat
on the Function. And I never have an issue running on my laptop.

Here's a simple Azure Function that fires lots of requests, one at a time, and illustrates this issue:

const cp = require('child_process');
const request = require('request');

module.exports = function (context, myTimer) {
context.log('Starting');

function doCall(cb) {
const url = 'https://jsonplaceholder.typicode.com/posts';
request(url, (err) => {
if (err) {
context.log("error: " + err.toString());
}

cb();
});
}

let i = 500;
doCall(function iterate() {
if (i-- > 0) {
context.log('iterate ' + i);
doCall(iterate);
} else {
context.log('done');
context.done();
}
});
};


I see it run successfully, time out for a couple minutes, run successfully again...

Answer Source

@david-ebbo, your suggestion was very helpful. From the link you posted, I can use an http Agent to limit the pool of sockets. It takes a while to run, but now no requests time out.

My example from above now looks like this:

const cp = require('child_process');
const request = require('request');
const http = require('http');

module.exports = function (context, myTimer) {
    context.log('Starting with agent');
    const pool = new http.Agent();
    pool.maxSockets = 4;

    function doCall(cb) {
        const url = 'https://jsonplaceholder.typicode.com/posts';
        request({ url, pool }, (err) => {
            if (err) {
                context.log("error: " + err.toString());
            }

            cb();
        });
    }

    let i = 500;
    doCall(function iterate() {
        if (i-- > 0) {
            context.log('iterate ' + i);
            doCall(iterate);
        } else {
            context.log('done');
            context.done();
        }
    });
};

Gaston's answer didn't help because I'm already only running one request at a time, so async.eachLimit won't make a difference.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download