dopplesoldner dopplesoldner - 4 months ago 19
Node.js Question

nodejs - mongodb native find all documents

Following an example from the mongodb manual for nodejs, I am finding all documents from a db as follows

mongo.Db.connect(mongoUri, function (err, db) {
if (err) {
console.log(err);
}
else {
db.collection('test').find().toArray(function(e, d) {
console.log(d.length);
db.close();
});
}
});


Now what I notice is that the entire set is converted to an array. As the dataset will grow, this will not be the ideal approach. Is there anyway to stream the data so it is not loaded in memory every time?

Thanks

Answer

The easiest way is to use a Cursor (reference):

var cursor = db.collection('test').find();

// Execute the each command, triggers for each document
cursor.each(function(err, item) {
    // If the item is null then the cursor is exhausted/empty and closed
    if(item == null) {
        db.close(); // you may not want to close the DB if you have more code....
        return;
    }
    // otherwise, do something with the item
});

If there's a lot of computation you need to do, you might consider whether a Map-Reduce (reference) would fit your needs as the code would execute on the DB server, rather than locally.