Thiago Arrais Thiago Arrais - 2 months ago 16
Node.js Question

Idiomatic way to wait for multiple callbacks in Node.js

Suppose you need to do some operations that depend on some temp file. Since
we're talking about Node here, those operations are obviously asynchronous.
What is the idiomatic way to wait for all operations to finish in order to
know when the temp file can be deleted?

Here is some code showing what I want to do:

do_something(tmp_file_name, function(err) {});
do_something_other(tmp_file_name, function(err) {});
fs.unlink(tmp_file_name);


But if I write it this way, the third call can be executed before the first two
get a chance to use the file. I need some way to guarantee that the first two
calls already finished (invoked their callbacks) before moving on without nesting
the calls (and making them synchronous in practice).

I thought about using event emitters on the callbacks and registering a counter
as receiver. The counter would receive the finished events and count how many
operations were still pending. When the last one finished, it would delete the
file. But there is the risk of a race condition and I'm not sure this is
usually how this stuff is done.

How do Node people solve this kind of problem?

Answer

Update:

Now I would advise to have a look at:

  • Promises

    The Promise object is used for deferred and asynchronous computations. A Promise represents an operation that hasn't completed yet, but is expected in the future.

    A popular promises library is bluebird. A would advise to have a look at why promises.

    You should use promises to turn this:

    fs.readFile("file.json", function (err, val) {
        if (err) {
            console.error("unable to read file");
        }
        else {
            try {
                val = JSON.parse(val);
                console.log(val.success);
            }
            catch (e) {
                console.error("invalid json in file");
            }
        }
    });
    

    Into this:

    fs.readFileAsync("file.json").then(JSON.parse).then(function (val) {
        console.log(val.success);
    })
    .catch(SyntaxError, function (e) {
        console.error("invalid json in file");
    })
    .catch(function (e) {
        console.error("unable to read file");
    });
    
  • generators: For example via co.

    Generator based control flow goodness for nodejs and the browser, using promises, letting you write non-blocking code in a nice-ish way.

    var co = require('co');
    
    co(function *(){
      // yield any promise
      var result = yield Promise.resolve(true);
    }).catch(onerror);
    
    co(function *(){
      // resolve multiple promises in parallel
      var a = Promise.resolve(1);
      var b = Promise.resolve(2);
      var c = Promise.resolve(3);
      var res = yield [a, b, c];
      console.log(res);
      // => [1, 2, 3]
    }).catch(onerror);
    
    // errors can be try/catched
    co(function *(){
      try {
        yield Promise.reject(new Error('boom'));
      } catch (err) {
        console.error(err.message); // "boom"
     }
    }).catch(onerror);
    
    function onerror(err) {
      // log any uncaught errors
      // co will not throw any errors you do not handle!!!
      // HANDLE ALL YOUR ERRORS!!!
      console.error(err.stack);
    }
    

If I understand correctly I think you should have a look at the very good async library. You should especially have a look at the series. Just a copy from the snippets from github page:

async.series([
    function(callback){
        // do some stuff ...
        callback(null, 'one');
    },
    function(callback){
        // do some more stuff ...
        callback(null, 'two');
    },
],
// optional callback
function(err, results){
    // results is now equal to ['one', 'two']
});


// an example using an object instead of an array
async.series({
    one: function(callback){
        setTimeout(function(){
            callback(null, 1);
        }, 200);
    },
    two: function(callback){
        setTimeout(function(){
            callback(null, 2);
        }, 100);
    },
},
function(err, results) {
    // results is now equals to: {one: 1, two: 2}
});

As a plus this library can also run in the browser.