Saurabh Saurabh - 29 days ago 8
JSON Question

JQ - removing duplicates using unique_by

I am trying to remove the duplicates from the following json by

id

Here is the json:

{
"Result": [
{
"name": "validation-of-art",
"id": "12",
"status": "passed",
"duration": 4740302
},
{
"name": "validation-of-art",
"id": "12",
"status": "passed",
"duration": 272320
},
{
"name": "validation-of-art",
"id": "13",
"status": "passed",
"duration": 272320
}
]
}


Here is what i have tried with:

jq -r 'unique_by(.Result.name)'


and also with
jq 'unique_by(.Result[].name)'


I am getting an error -
Cannot index array with string "Result"

Any help would be appreciated.

Answer Source

Here is an example which eliminates all but one of the .Result objects using unique_by(.name)

$ jq -M '.Result |= unique_by(.name)' data.json
{
  "Result": [
    {
      "name": "validation-of-art",
      "id": "12",
      "status": "passed",
      "duration": 4740302
    }
  ]
}

If this isn't quite what you want you can generalize this easily. E.g. to keep one object for each unique {name,id} you could use

$ jq -M '.Result |= unique_by({name, id})' data.json
{
  "Result": [
    {
      "name": "validation-of-art",
      "id": "12",
      "status": "passed",
      "duration": 4740302
    },
    {
      "name": "validation-of-art",
      "id": "13",
      "status": "passed",
      "duration": 272320
    }
  ]
}