unconditionalcoder unconditionalcoder - 1 year ago 825
Bash Question

Parse JSON to array in shell script

I'm trying to parse a JSON object within a shell script into an array.

e.g.: [Amanda, 25, http://mywebsite.com]

The JSON looks like:

"name" : "Amanda",
"age" : "25",
"websiteurl" : "http://mywebsite.com"

I do not want to use any libraries, it would be best if I could use a regular expression or grep. I have done:

myfile.json | grep name

This gives me "name" : "Amanda". I could do this in a loop for each line in the file, and add it to an array but I only need the right side and not the entire line

Answer Source

If you really cannot use a proper JSON parser such as jq[1] , try an awk-based solution:

Bash 4.x:

readarray -t values < <(awk -F\" 'NF>=3 {print $4}' myfile.json)

Bash 3.x:

IFS=$'\n' read -d '' -ra values < <(awk -F\" 'NF>=3 {print $4}' myfile.json)

This stores all property values in Bash array ${values[@]}, which you can inspect with
declare -p values.

These solutions have limitations:

  • each property must be on its own line,
  • all values must be double-quoted,
  • embedded escaped double quotes are not supported.

All these limitations reinforce the recommendation to use a proper JSON parser.

Note: The following alternative solutions use the Bash 4.x+ readarray -t values command, but they also work with the Bash 3.x alternative, IFS=$'\n' read -d '' -ra values.

grep + cut combination: A single grep command won't do (unless you hard-code assumptions about whitespace or column positions - see below), but adding cut helps:

readarray -t values < <(grep '"' myfile.json | cut -d '"' -f4)

GNU grep: If you are willing to assume that there's exactly one space between the : and the opening " of the property value:

readarray -t values < <(grep -Po '(?<=: ").+(?="\s*,?\s*$)' myfile.json)

Finally, here's a pure Bash (3.x+) solution:

What makes this as a viable alternative in terms of performance is that no external utilities are called in each loop iteration; however, for larger input files a solution based on external utilities will be faster.

#!/usr/bin/env bash

declare -a values # declare the array                                                                                                                                                                  

# Read each line and parse into fields by double quotes.
# The 3rd field will contain everything after the property name on each line.
while IFS=\" read unused unused valuePart; do
  # If there is a value part, extract the value from between the double quotes
  # and add it to the array.
  [[ $valuePart =~ \"(.*)\" ]] && values+=( "${BASH_REMATCH[1]}" )
done < myfile.json                                                                                                                                          

declare -p values # print the array

[1] Here's what a robust jq-based solution would look like (Bash 4.x):
readarray -t values < <(jq -r '(map(values))[]' myfile.json)

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download