Darpan Darpan - 1 year ago 81
Python Question

Save a subset of MongoDB(3.0) collection to another collection in Python

I found this answer - Answer link

db.full_set.aggregate([ { $match: { date: "20120105" } }, { $out: "subset" } ]);

I want do same thing but with first 15000 documents in collection, I couldn't find how to apply limit to such query (I tried using
$limit : 15000
, but it doesn't recognize $limit)

also when I tried -


there is no function
for output type

Guide me how can I accomplish it?

Answer Source

in python, this is how things work - $limit needs to be wrapped in "",
and you need to create a pipeline to execute it as a command.

In my code -

    pipeline = [{ '$limit': 15000 },{'$out': "destination_collection"}]
    db.command('aggregate', "source_collection", pipeline=pipeline)

You need to wrap everything in double quotes, including your source and destination collection. And in db.command db is the object of your database (ie dbclient.database_name)

As per this answer -

It works about 100 times faster than forEach at least in my case. This is because the entire aggregation pipeline runs in the mongod process, whereas a solution based on find() and insert() has to send all of the documents from the server to the client and then back. This has a performance penalty, even if the server and client are on the same machine.

The one that really helped me figure this answer out - Reference 1
And official documentation

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download