Henry Barber Henry Barber - 9 months ago 38
Python Question

How to find a Cloudant record missing a field

I've got a database with records that look like this

"_id": "eaed1a721cd763c68f7db76862cbf2c1",
"_rev": "4-2f99dc06018a72af40143b47f0825aaa",
"categorisationComments": "null",
"description": "A mobile image-, video-sharing and social network service",
"appName": "Instagram",
"baseUrl": "instagram.com",
"appId": 67,
"currentStatus": 3,
"primaryCategory": "Social Media"

the problem I have, is there's about 7000 of them and it takes a long while to read through them all with Python and couchdb.
When I loaded the data the appId was assigned a value (for no good reason at all) of -1 ; this was a poor choice.
Because this is were my Python script is choking

if doc['appId'] == -1: KeyError: 'appId'

so, I seem to have a record that is missing the appId field.

How can I create a Cloudant query or a couchdb search to find my wayward record ? What I really want is a query because it takes about 30 minutes for couchdb to read every line

Answer Source

In the Cloudant dashboard, click New View and add a "map" function like so

function(doc) {
  if (typeof doc.appId === 'undefined') {
    emit(doc._id, null);

Cloudant will feed all of the documents in your database through this function. Any with a missing 'appId' property will form an index containing the document's id and 'null'. After the index has built (a few seconds) you should see any documents with missing ids on the right-hand side of the window.

See https://docs.cloudant.com/creating_views.html for more details on creating MapReduce views in Cloudant.