Rob Rob - 1 year ago 99
Python Question

how to speed up mongoengine queries

If I have two objects like:

class User(Document):
name = StringField()
following = ListField(ReferenceField('User'))
meta = {
'indexes': [

class Media(Document):
owner = ReferenceField('User')
url = StringField()
is_hidden = BooleanField()
posted_date = Date

meta = {
'indexes': [

and when I want to look up the following condition where it is not hidden and its owner is someone I am following and it was recently posted, I have a query like this:

user = User.objects.first()
Media.objects(Q(owner__in=user.following) &
Q(is_hidden=False) &

which is not scaling and is becoming much slower. What can I do to speed up the performance of these types of complex queries?

Answer Source

1) Use User.objects.get(id=user_id) instead of first(). Tough I am not really sure if that will make a difference, I think yes and that is a find operation where MongoDB will return a cursor and mongoengine iterates to the first document. Instead, get() is doing a findOne and returns only 1 document. If I mistake, someone should correct me please.

2) Use a compound index (not multiple single index) since your query is using multiple fields (example here also):

 meta = {
            'indexes': [
                ('owner', 'posted_date', 'is_hidden',)

3) Limit returning data to only the fields you need, project your fields, using only()

Use explain() on your query to styudy it and improve it in oder to reach a covered query.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download