Sylvain Sylvain - 3 months ago 23
Bash Question

How to select and merge objects data with jq?

After some hours playing with jq, I need your help for selecting and merging objects.

I have this kind of JSON :

{
"cluster-1": {
"vhosts": {
"vhost_aaa": {
"postgresql": {
"bdd1": {
"db_aaa": {
"user_aaa": {
"password": "xxx"
}
}
},
"bdd2": {
"db_aaa": {
"user_aaa": {
"password": "xxx"
}
}
}
}
}
}
},
"cluster-2": {
"vhosts": {
"vhost_bbb": {
"postgresql": {
"bdd1": {
"db_bbb": {
"user_bbb": {
"password": "xxx"
},
"user_bbb_ro": {
"password": "xxx"
}
}
}
}
},
"vhost_ccc": {
"postgresql": {
"bdd1": {
"db_ccc": {
"user_ccc": {
"password": "xxx"
}
}
}
}
}
}
}
}


This is a deep JSON which tell me that, on cluster-x there is vhosts (vhosts_xxx) which may use a kind of database (here postgresql) which is hosted by a server (here bdd1 or bdd2) which contains databases (db_xxx) where i want to create specified users (user_xxx) with credentials details. Ouf !

My goal is to select part of data to make specific actions, so for databases servers, I want all actions to do on the same server so this is what I try to generate with jq :

"bdd1": {
"db_aaa": {
"user_aaa": {
"password": "xxx"
}
},
"db_bbb": {
"user_bbb": {
"password": "xxx"
},
"user_bbb_ro": {
"password": "xxx"
}
},
"db_ccc": {
"user_ccc": {
"password": "xxx"
}
}
},
"bdd2": {
"db_aaa": {
"user_aaa": {
"password": "xxx"
}
}
}


With this filter
(..|.vhosts?|..|.postgresql?)|objects
, I can isolate data I need.

{
"bdd1": {
"db_aaa": {
"user_aaa": {
"password": "xxx"
}
}
},
"bdd2": {
"db_aaa": {
"user_aaa": {
"password": "xxx"
}
}
}
}
{
"bdd1": {
"db_bbb": {
"user_bbb": {
"password": "xxx"
},
"user_bbb_ro": {
"password": "xxx"
}
}
}
}
{
"bdd1": {
"db_ccc": {
"user_ccc": {
"password": "xxx"
}
}
}
}


Next step is to merge all of theses data grouped by bddx.
Any help or advice will be welcome :)

Answer

You can use to_entries to convert an object to an array of key/value pairs of properties/values. You could then group those pairs to construct your result.

[..|.vhosts?|..|.postgresql?|objects|to_entries[]]
    | reduce group_by(.key)[] as $g ({};
        .[$g[0].key] = [$g[].value]
    )

This yields the following result:

{
  "bdd1": [
    {
      "db_aaa": {
        "user_aaa": { "password": "xxx" }
      }
    },
    {
      "db_ccc": {
        "user_ccc": { "password": "xxx" }
      }
    },
    {
      "db_bbb": {
        "user_bbb": { "password": "xxx" },
        "user_bbb_ro": { "password": "xxx" }
      }
    }
  ],
  "bdd2": [
    {
      "db_aaa": {
        "user_aaa": { "password": "xxx" }
      }
    }
  ]
}