dopplesoldner dopplesoldner - 2 months ago 6x
Node.js Question

nodejs - mongodb native find all documents

Following an example from the mongodb manual for nodejs, I am finding all documents from a db as follows

mongo.Db.connect(mongoUri, function (err, db) {
if (err) {
else {
db.collection('test').find().toArray(function(e, d) {

Now what I notice is that the entire set is converted to an array. As the dataset will grow, this will not be the ideal approach. Is there anyway to stream the data so it is not loaded in memory every time?



The easiest way is to use a Cursor (reference):

var cursor = db.collection('test').find();

// Execute the each command, triggers for each document
cursor.each(function(err, item) {
    // If the item is null then the cursor is exhausted/empty and closed
    if(item == null) {
        db.close(); // you may not want to close the DB if you have more code....
    // otherwise, do something with the item

If there's a lot of computation you need to do, you might consider whether a Map-Reduce (reference) would fit your needs as the code would execute on the DB server, rather than locally.