Converting JSON response to key value pair using jq

Related searches

So, I am getting a response from an API that I am calling in a shell script in the following form

[{"id":100000004,"name":"Customs Clearance Requested"},{"id":100000005,"name":"Customs Cleared"},{"id":100000006,"name":"Cargo Loaded to Vessel"}]

I want to create a map out of it that will help me lookup the id's from a name and use it in the shell script. So something like map["Customs Clearance Requested"] would give me 100000004 which I can use further. Can this be done using jq? I am pretty new to shell scripting and jq and got stuck with above thing

json='[{"id":100000004,"name":"Customs Clearance Requested"},{"id":100000005,"name":"Customs Cleared"},{"id":100000006,"name":"Cargo Loaded to Vessel"}]'

declare -A map
while IFS= read -r -d '' name && IFS= read -r -d '' value; do
  map[$name]=$value
done < <(jq -j '.[] | "\(.name)\u0000\(.id)\u0000"' <<<"$json")

declare -p map  # demo purposes: print the map we created as output

...emits as output:

declare -A map=(["Cargo Loaded to Vessel"]="100000006" ["Customs Clearance Requested"]="100000004" ["Customs Cleared"]="100000005" )

...which you can query exactly as requested:

$ echo "${map['Cargo Loaded to Vessel']}"
100000006

Converting JSON response to key value pair using jq, A series of how to examples on using jq, a command-line JSON processor. To find a key and value jq can filter based on keys and return the value. Suppose we have the jq can be used for more than just reading values from a JSON object. It can also transform JSON into new data structures. Returning� We can use jq‘s delete function, del(), to delete a key:value pair. To delete the message key:value pair, we type this command: jq "del(.message)" iss.json. Note this doesn’t actually delete it from the “iss.json” file; it just removes it from the output of the command. If you need to create a new file without the message key:value pair

You could use the select function, e.g.:

data='[{"id":100000004,"name":"Customs Clearance Requested"},{"id":100000005,"name":"Customs Cleared"},{"id":100000006,"name":"Cargo Loaded to Vessel"}]'
jq 'map(select(.["name"] == "Customs Clearance Requested"))' <<< $data

It will get all elements which name equals "Customs Clearance Requested", e.g.:

[
  {
    "id": 100000004,
    "name": "Customs Clearance Requested"
  }
]

If you want to get the id field:

jq 'map(select(.["name"] == "Customs Clearance Requested")["id"])' <<< $data

This will output:

[
  100000004
]

Please note that it will return an array and not a single element because the search does not know how many results will be found.

If you want to generalize this in a shell function, you could write:

function get_id_from_name
{
  # $1=name to search for
  local filter=$(printf 'map(select(.["name"] == "%s")["id"])' "$1")
  jq "$filter"
}

Then call it like that:

get_id_from_name "Customs Clearance Requested" <<< $data

If your data is stored in a file, you could call it this way:

get_id_from_name "Customs Clearance Requested" < /path/to/file.json

JSON on the command line with jq, Most of the popular API and data services use the JSON data format, so we'll interesting information, and how to use the jq to parse it at the command-line. JSON stands for JavaScript Object Notation and is nearly ubiquitous as a data structures known as Objects consisting of attribute-value pairs. JSON objects are surrounded with curly braces {} and contain key-value pairs. The keys must be of type:string but the values can be any valid JSON type. This includes string, number, object, array

The following is very similar to @CharlesDuffy's excellent answer but does not assume that the .name and .id values are NUL-free (i.e., do not have any "\u0000" characters):

declare -A map
while read -r name
do
    name=$(sed -e 's/^"//' -e 's/"$//' <<< "$name")
    read -r id
    map[$name]="$id"
done < <(echo "$json" | jq -c '.[]|.name,.id')

The point is that the -j option is like -r (i.e., produces "raw output"), whereas the -c option produces JSON.

This means that if you don't want the .id values as JSON strings, then the above won't be a solution; also, if the .name values contain double-quotes, then you might want to deal with the occurrences of \".

Parsing JSON with jq, jq is a powerful tool that lets you read, filter, and write JSON in bash. You have to pipe to 4 different utilities just to get to a property in the JSON response body! Bash doesn't an object, .[] will output the value of each key/value pair: We'll create a script that uses imagemagick to bulk convert HEIC files� That’s tough to read and even tougher to write. You have to pipe to 4 different utilities just to get to a property in the JSON response body! Bash doesn’t understand JSON out of the box, and using the typical text manipulation tools like grep, sed, or awk, gets difficult. Luckily there’s a better way using a tool called jq.

Working with JSON in bash using jq, Using JQ to Convert JSON to CSV: Top Level Values. #jq #json #shopify billing address (type: object with 15 properties, aka key-value pairs). If you send functions using JSON, the functions will lose their scope, and the receiver would have to use eval() to convert them back into functions. Browser Support The JSON.stringify() function is included in JavaScript (ECMAScript) and is supported in all major browsers.

Using JQ to Convert JSON to CSV: Top Level Values, JSON stands for JavaScript Object Notation and is nearly ubiquitous as a data format, for its lightweight nature and (relatively) ease of human-readability. Using jq. jq is a program described as "sed for JSON data":

A JSON object is a key-value data format that is typically rendered in curly braces. JSON object consist of curly braces ({ }) at the either ends and have key-value pairs inside the braces. Each key-value pair inside braces are separated by comma (, ). JSON object looks something like this :

Comments
  • This is very closely related to converting a json object into a bash associative array, maybe even an outright duplicate (unless you don't know how to use jq to convert your current JSON format into the one that question's answers use as input).
  • Calling jq once per item you want to retrieve seems a little unfortunate, vs converting the whole document to a single bash object all at once.
  • And injecting data into strings parsed as code (in any language) is a bad idea. If you want to pass variables to jq, that's what --arg is for. Thus, something like: get_id_from_name() { jq -r --arg name "$1" 'map(select(.["name"] == $name)["id"]) | .[]'; }
  • @CharlesDuffy I assumed that the OP wanted a single result, but now that you mention it I think you’re right that he needs a map built on the bash side.
  • echo $json is not particularly safe -- if that JSON contains "Hello * World", then the * will be replaced with a list of filenames in the current directory. Honestly, if there's a need to handle content with NULs safely, the better answer is to filter them out in jq -- remember that bash uses C strings and can't represent literal NULs at all.
  • ...the code here would be doubling up literal backslashes, representing tabs as two-character \t sequences, and otherwise not be faithfully representing the JSON values as C strings on the shell side.
  • See the last code snippet in stackoverflow.com/a/51234929/14122 for an example of an answer being deliberately cautious about NUL handling.