natureminded natureminded - 4 years ago 131
AngularJS Question

How do I deal with nested mongoose queries and async issues in my one-to-many relationship?

I'm trying to query for all posts from a database, and then grab all comments belonging to each post, and send that whole thing back to the front end to display. My strategy thus far has been using nested Mongoose queries (see pseudo-code and actual code examples below), and am getting some unexpected results due to async issues.

Can anyone tell me where I'm going wrong, or if there's a better way to do what I'm trying to accomplish:

My Schemas:



I have three
Schemas
in Mongoose:


  1. UserSchema (
    User
    )

  2. PostSchema (
    Post
    )

  3. CommentSchema (
    PostComment
    )



I've only included the
CommentSchema
here, to simplify my question:

var CommentSchema = new mongoose.Schema (
{
message: {
type: String,
minlength: [2, 'Your comment must be at least 2 characters.'],
maxlength: [2000, 'Your comment must be less than 2000 characters.'],
required: [true, 'You cannot submit an empty comment.'],
trim: true,
}, // end message field
userID: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
username: {
type: String,
},
postID: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Post'
},
},
{
timestamps: true,
}
);


When a new Comment is created, the
_id
of the post is recorded into the
.postID
field of the comment.

My Pseudo-Code Strategy:



// query for all posts using a mongoose promise
// run a `for loop` through the array returned posts
// query my comments collection for any comments pertaining to post[i]
// attach comments returned to the post[i]
// push post[i] into a new array (now with comments attached)
// check if on last run through array
// res.json the new array back to the front end
// On front end just iterate through each post and its contained comments.


However, when I attempt this strategy, I get some async issues with my second Mongoose query within the for loop.

My Actual Code Example:



Post.find({}) // first query
.then(function(allPosts) {
for (var i = 0; i < allPosts.length; i++) {
_post = allPosts[i];
console.log(_post, i);
PostComment.find({postID: _post._id}) // **nested query
.then(function(comments) {
console.log({
post: _post, // this value will not match the console log above (outside of the query)
index_value: i, // this value too will be out of match with the console log above
comments: comments,
});
// attach new comment to allPosts[i]
// push post with comment attached to new array
// check if on last iteration, if so res.json new array
})
.catch(function(err) {
console.log(err);
})
}
.catch(function(err) {
console.log(err);
}


Issue from Code Example Above:



In the example above, in the second query, the
**nested query
, the value of
i
and
_post
are out of sync by the time the data is returned from the mongoose promise (the
.then
). The for loop is progressing faster than the data is being returned. Thus, if I try and attach any comments to the parent post object (
_post
), the variable is already out of synch with the progression of the for loop (
_post
now becomes the next post in the array). I'm stumped on how to remedy this and get all my comments from each post, and bundle this together for the front end. I'm just confused at this point.

Desired Behavior:



I want a populated list of all my posts, with the comments attached to each post, to make iterating over them on the front end easier. That way on the front end, all posts are shown with their respective comments beneath them.

Conclusion:



What am I doing wrong? How can I iterate through all of my posts, and grab all the comments for each, and make it tidy for front-end display in Angular? Is my approach wrong or too "costly" in my queries? Is there a better way to achieve my desired behavior?

Any insight or help is beyond appreciated! I searched around hoping to see another question like this, and have been hitting my head on this issue for awhile =)

Answer Source

I figured out how to accomplish this. I did have to make some changes to my Schemas. Using the populate() Mongoose method (http://mongoosejs.com/docs/populate.html)

Keeping this explanation short, I wanted to show the updated Schemas, and then show how I was able to populate.

PostSchema:

// Setup dependencies:
var mongoose = require('mongoose'),
    Schema = mongoose.Schema;

// Setup a schema:
var PostSchema = new Schema (
    {
        message: {
            type: String,
            minlength: [2, 'Your post must be at least 2 characters.'],
            maxlength: [2000, 'Your post must be less than 2000 characters.'],
            required: [true, 'You cannot submit an empty post.'],
            trim: true,
        }, // end message field
        _user: {
            type: Schema.Types.ObjectId,
            ref: 'User'
        },
        username: {
            type: String,
        },
        comments: [{
            type: Schema.Types.ObjectId,
            ref: 'Comment'
        }],
    },
    {
        timestamps: true,
    }
);

// updates userID based upon current session login info:
PostSchema.methods.updateUserID = function(id) {
    this._user = id;
    this.save();
    return true;
};

// updates username based on found username based on session ID:
PostSchema.methods.updateUsername = function(username) {
    this.username = username;
    this.save();
    return true;
};

// adds comment to post's comments array:
PostSchema.methods.addComment = function(commentID) {
    this.comments.push(commentID);
    this.save();
    return true;
};

// Instantiate our model and export it:
module.exports = mongoose.model('Post', PostSchema);

CommentSchema:

// Setup dependencies:
var mongoose = require('mongoose'),
    Schema = mongoose.Schema;

// Setup a schema:
var CommentSchema = new Schema (
    {
        message: {
            type: String,
            minlength: [2, 'Your comment must be at least 2 characters.'],
            maxlength: [2000, 'Your comment must be less than 2000 characters.'],
            required: [true, 'You cannot submit an empty comment.'],
            trim: true,
        }, // end message field
        _user: {
            type: Schema.Types.ObjectId,
            ref: 'User'
        },
        username: {
            type: String,
        },
        _post: {
            type: Schema.Types.ObjectId,
            ref: 'Post'
        },
    },
    {
        timestamps: true,
    }
);

// Assigns user ID to comment when called (uses session info):
CommentSchema.methods.updateUserID = function(id) {
    this._user = id;
    this.save();
    return true;
};

// Assigns post ID to comment when called:
CommentSchema.methods.updatePostID = function(id) {
    this._post = id;
    this.save();
    return true;
};

// Sets username field to comment when called (this is not best practice):
CommentSchema.methods.updateUsername = function(username) {
    this.username = username;
    this.save();
    return true;
};

// Instantiate our model and export it:
module.exports = mongoose.model('Comment', CommentSchema);

Solution: Use the Populate Method:

Using the populate() method (http://mongoosejs.com/docs/populate.html) we can start with our Post model and populate the comments field.

Using the instance methods defined in the PostSchema, we push comment ID's into the Post.comments array, which the populate() method below grabs all comment objects and replaces the IDs with the actual comment objects.

var User = require('mongoose').model('User');
var Post = require('mongoose').model('Post');
var PostComment = require('mongoose').model('Comment');

Post.find({})
    .populate('comments')  // populates comments objects based on ids in `comments`
    .exec()
    .then(function(commentsAndPosts) {
        console.log(commentsAndPosts);
        return res.json(commentsAndPosts);
    })
    .catch(function(err) {
        console.log(err);
        return res.json(err);
    })

The link provided to Mongoose's documentation has a nice clean example. If any questions, please let me know.

But now, on the front end, I can access the comments array inside each post object and all comments have been populated! Sweet!

Summary and Notes:

I was able to store an array of comment IDs inside each Post object. Thus, using populate(), we can query into the Comments collection and grab all comment objects associated with the relevant ID. This is great, as after our populate method completes, we can send back our entire array of all posts and comments as one JSON object, and iterate over them on the front-end.

There are some "bad practices" in my database design above. In my PostSchema and CommentSchema above, I have username fields in each model, which stores the respective creator of each object. This is necessary, as this data can be obtained by querying the database based upon the _user field (which is the _id of the user whom created the post or comment). This mistake in design was something I did as I was trying to get desired functionality. I thought maybe it might allow for less queries to the DB and perhaps speed up rendering the posts and comments on the front-end (I'd be curious if anyone had any insight to their thoughts on that, but might be for another question).

In truth, I am going to attempt to modify my codebase next so that the usernames are populated in a similar manner to the example above, rather than create an extra and unnecessary field. But for now, I was able to query into my associated collections without running into any async issues, and return a nice JSON object with everything to the front-end.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download