crashfocus crashfocus - 1 year ago 87
JSON Question

Read from large MongoDB, write to JSON

I have a MongoDB with about 70gb of data in it. I need to download the entire DB into one JSON document, preferably with Python. However, when I try and do

json.dump(object, file_object)
a couple million times, I tend to unintentionally create a malformed JSON. Storing the entire DB in memory isn't an option, because, well, 70gb > 16gb of RAM. How can I write JSON documents, one at a time, to the same file object in Python?


Answer Source

Use mongoexport to export the data:

mongoexport --db test --collection traffic --out traffic.json
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download