mjhjmtu

I've got to extract a license plate digit and it's associated confidence from a JSON string that looks like:

 "response": 
 "container": 
 "id": "0df307bc-06b2-45cf-b7ff-ce07fd04e04d",
 "timestamp": "2018-Jul-10 17:34:27.448632"
 ,
 "id": "00000002-0000-0000-0000-000000000015"
 ,
 "frames": 
 "frame": 
 "id": "5583",
 "timestamp": "2016-Nov-30 13:05:27",
 "lps": 
 "lp": 
 "licenseplate": "15451BBL",
 "text": "15451BBL",
 "wtext": "15451BBL",
 "confidence": "20",
 "bkcolor": "16777215",
 "color": "16777215",
 "type": "0",
 "ntip": "11",
 "cct_country_short": "",
 "cct_state_short": "",
 "tips": 
 "tip": 
 "poly": 
 "p": "x": "1094", "y": "643" ,
 "p": "x": "1099", "y": "643" ,
 "p": "x": "1099", "y": "667" ,
 "p": "x": "1094", "y": "667" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "49",
 "code_ascii": "1",
 "confidence": "97"
 ,
 "tip": 
 "poly": 
 "p": "x": "1103", "y": "642" ,
 "p": "x": "1113", "y": "642" ,
 "p": "x": "1112", "y": "667" ,
 "p": "x": "1102", "y": "667" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "53",
 "code_ascii": "5",
 "confidence": "89"
 ,
 "tip": 
 "poly": 
 "p": "x": "1112", "y": "640" ,
 "p": "x": "1122", "y": "640" ,
 "p": "x": "1122", "y": "666" ,
 "p": "x": "1112", "y": "666" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "52",
 "code_ascii": "4",
 "confidence": "97"
 ,
 "tip": 
 "poly": 
 "p": "x": "1123", "y": "640" ,
 "p": "x": "1132", "y": "640" ,
 "p": "x": "1131", "y": "665" ,
 "p": "x": "1123", "y": "665" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "53",
 "code_ascii": "5",
 "confidence": "97"
 ,
 "tip": 
 "poly": 
 "p": "x": "1134", "y": "640" ,
 "p": "x": "1139", "y": "640" ,
 "p": "x": "1139", "y": "664" ,
 "p": "x": "1133", "y": "664" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "49",
 "code_ascii": "1",
 "confidence": "77"
 ,
 "tip": 
 "poly": 
 "p": "x": "1154", "y": "639" ,
 "p": "x": "1163", "y": "639" ,
 "p": "x": "1163", "y": "663" ,
 "p": "x": "1153", "y": "663" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "66",
 "code_ascii": "B",
 "confidence": "97"
 ,
 "tip": 
 "poly": 
 "p": "x": "1164", "y": "638" ,
 "p": "x": "1173", "y": "638" ,
 "p": "x": "1173", "y": "663" ,
 "p": "x": "1163", "y": "663" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "66",
 "code_ascii": "B",
 "confidence": "94"
 ,
 "tip": 
 "poly": 
 "p": "x": "1191", "y": "637" ,
 "p": "x": "1206", "y": "636" ,
 "p": "x": "1205", "y": "660" ,
 "p": "x": "1190", "y": "661" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "76",
 "code_ascii": "L",
 "confidence": "34"
 ,
 "tip": 
 "poly": 
 "p": "x": "1103", "y": "655" ,
 "p": "x": "1111", "y": "655" ,
 "p": "x": "1111", "y": "667" ,
 "p": "x": "1103", "y": "667" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "74",
 "code_ascii": "J",
 "confidence": "57"
 ,
 "tip": 
 "poly": 
 "p": "x": "1103", "y": "655" ,
 "p": "x": "1111", "y": "655" ,
 "p": "x": "1111", "y": "667" ,
 "p": "x": "1103", "y": "667" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "74",
 "code_ascii": "J",
 "confidence": "57"
 ,
 "tip": 
 "poly": 
 "p": "x": "1176", "y": "638" ,
 "p": "x": "1185", "y": "637" ,
 "p": "x": "1184", "y": "661" ,
 "p": "x": "1175", "y": "662" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "52",
 "code_ascii": "4",
 "confidence": "7"
 
 ,
 "ncharacter": "8",
 "characters": 
 "characater": 
 "poly": 
 "p": "x": "1094", "y": "643" ,
 "p": "x": "1099", "y": "643" ,
 "p": "x": "1099", "y": "667" ,
 "p": "x": "1094", "y": "667" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "49",
 "code_ascii": "1",
 "confidence": "97"
 ,
 "characater": 
 "poly": 
 "p": "x": "1103", "y": "642" ,
 "p": "x": "1113", "y": "642" ,
 "p": "x": "1112", "y": "667" ,
 "p": "x": "1102", "y": "667" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "53",
 "code_ascii": "5",
 "confidence": "89"
 ,
 "characater": 
 "poly": 
 "p": "x": "1112", "y": "640" ,
 "p": "x": "1122", "y": "640" ,
 "p": "x": "1122", "y": "666" ,
 "p": "x": "1112", "y": "666" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "52",
 "code_ascii": "4",
 "confidence": "97"
 ,
 "characater": 
 "poly": 
 "p": "x": "1123", "y": "640" ,
 "p": "x": "1132", "y": "640" ,
 "p": "x": "1131", "y": "665" ,
 "p": "x": "1123", "y": "665" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "53",
 "code_ascii": "5",
 "confidence": "97"
 ,
 "characater": 
 "poly": 
 "p": "x": "1134", "y": "640" ,
 "p": "x": "1139", "y": "640" ,
 "p": "x": "1139", "y": "664" ,
 "p": "x": "1133", "y": "664" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "49",
 "code_ascii": "1",
 "confidence": "77"
 ,
 "characater": 
 "poly": 
 "p": "x": "1154", "y": "639" ,
 "p": "x": "1163", "y": "639" ,
 "p": "x": "1163", "y": "663" ,
 "p": "x": "1153", "y": "663" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "66",
 "code_ascii": "B",
 "confidence": "97"
 ,
 "characater": 
 "poly": 
 "p": "x": "1164", "y": "638" ,
 "p": "x": "1173", "y": "638" ,
 "p": "x": "1173", "y": "663" ,
 "p": "x": "1163", "y": "663" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "66",
 "code_ascii": "B",
 "confidence": "94"
 ,
 "characater": 
 "poly": 
 "p": "x": "1191", "y": "637" ,
 "p": "x": "1206", "y": "636" ,
 "p": "x": "1205", "y": "660" ,
 "p": "x": "1190", "y": "661" 
 ,
 "bkcolor": "16777215",
 "color": "0",
 "code": "76",
 "code_ascii": "L",
 "confidence": "34"
 
 ,
 "det_time_us": "1104009",
 "poly": 
 "p": "x": "1088", "y": "642" ,
 "p": "x": "1210", "y": "634" ,
 "p": "x": "1210", "y": "661" ,
 "p": "x": "1087", "y": "669" 
 
 
 ,
 "det_time_us": "1710270"
 
 

I've got something like:

$ jq -r '.frames.frame.lps.lp|.characters.characater.code_ascii,.characters.characater.confidence' test.json

but it only returns one single letter & it's conf. score....

Question

• How can I get all letters and the associated score back?

Output

I expect the output to be like:

1 97, 5 89, 4 97,5 97, 1 77,B 97, B 94, L 34, J, 57,J 57, 4 7, 1 97, 5 89, 4 97, 5 97, 1 77, B 97, B 94, L 34

NOTE: format can be different, this is just indicating the data I wanted to extract.

copy & paste

input file "test.json"

"response":"container":"id":"41d6efcb-24d6-490d-8880-762255519b5f","timestamp":"2018-Jul-11 19:51:06.461665","id":"00000002-0000-0000-0000-000000000015","frames":"frame":"id":"5583","timestamp":"2016-Nov-30 13:05:27","lps":"lp":"licenseplate":"15451BBL","text":"15451BBL","wtext":"15451BBL","confidence":"20","bkcolor":"16777215","color":"16777215","type":"0","ntip":"11","cct_country_short":"","cct_state_short":"","tips":"tip":"poly":"p":"x":"1094","y":"643","p":"x":"1099","y":"643","p":"x":"1099","y":"667","p":"x":"1094","y":"667","bkcolor":"16777215","color":"0","code":"49","code_ascii":"1","confidence":"97","tip":"poly":"p":"x":"1103","y":"642","p":"x":"1113","y":"642","p":"x":"1112","y":"667","p":"x":"1102","y":"667","bkcolor":"16777215","color":"0","code":"53","code_ascii":"5","confidence":"89","tip":"poly":"p":"x":"1112","y":"640","p":"x":"1122","y":"640","p":"x":"1122","y":"666","p":"x":"1112","y":"666","bkcolor":"16777215","color":"0","code":"52","code_ascii":"4","confidence":"97","tip":"poly":"p":"x":"1123","y":"640","p":"x":"1132","y":"640","p":"x":"1131","y":"665","p":"x":"1123","y":"665","bkcolor":"16777215","color":"0","code":"53","code_ascii":"5","confidence":"97","tip":"poly":"p":"x":"1134","y":"640","p":"x":"1139","y":"640","p":"x":"1139","y":"664","p":"x":"1133","y":"664","bkcolor":"16777215","color":"0","code":"49","code_ascii":"1","confidence":"77","tip":"poly":"p":"x":"1154","y":"639","p":"x":"1163","y":"639","p":"x":"1163","y":"663","p":"x":"1153","y":"663","bkcolor":"16777215","color":"0","code":"66","code_ascii":"B","confidence":"97","tip":"poly":"p":"x":"1164","y":"638","p":"x":"1173","y":"638","p":"x":"1173","y":"663","p":"x":"1163","y":"663","bkcolor":"16777215","color":"0","code":"66","code_ascii":"B","confidence":"94","tip":"poly":"p":"x":"1191","y":"637","p":"x":"1206","y":"636","p":"x":"1205","y":"660","p":"x":"1190","y":"661","bkcolor":"16777215","color":"0","code":"76","code_ascii":"L","confidence":"34","tip":"poly":"p":"x":"1103","y":"655","p":"x":"1111","y":"655","p":"x":"1111","y":"667","p":"x":"1103","y":"667","bkcolor":"16777215","color":"0","code":"74","code_ascii":"J","confidence":"57","tip":"poly":"p":"x":"1103","y":"655","p":"x":"1111","y":"655","p":"x":"1111","y":"667","p":"x":"1103","y":"667","bkcolor":"16777215","color":"0","code":"74","code_ascii":"J","confidence":"57","tip":"poly":"p":"x":"1176","y":"638","p":"x":"1185","y":"637","p":"x":"1184","y":"661","p":"x":"1175","y":"662","bkcolor":"16777215","color":"0","code":"52","code_ascii":"4","confidence":"7","ncharacter":"8","characters":"characater":"poly":"p":"x":"1094","y":"643","p":"x":"1099","y":"643","p":"x":"1099","y":"667","p":"x":"1094","y":"667","bkcolor":"16777215","color":"0","code":"49","code_ascii":"1","confidence":"97","characater":"poly":"p":"x":"1103","y":"642","p":"x":"1113","y":"642","p":"x":"1112","y":"667","p":"x":"1102","y":"667","bkcolor":"16777215","color":"0","code":"53","code_ascii":"5","confidence":"89","characater":"poly":"p":"x":"1112","y":"640","p":"x":"1122","y":"640","p":"x":"1122","y":"666","p":"x":"1112","y":"666","bkcolor":"16777215","color":"0","code":"52","code_ascii":"4","confidence":"97","characater":"poly":"p":"x":"1123","y":"640","p":"x":"1132","y":"640","p":"x":"1131","y":"665","p":"x":"1123","y":"665","bkcolor":"16777215","color":"0","code":"53","code_ascii":"5","confidence":"97","characater":"poly":"p":"x":"1134","y":"640","p":"x":"1139","y":"640","p":"x":"1139","y":"664","p":"x":"1133","y":"664","bkcolor":"16777215","color":"0","code":"49","code_ascii":"1","confidence":"77","characater":"poly":"p":"x":"1154","y":"639","p":"x":"1163","y":"639","p":"x":"1163","y":"663","p":"x":"1153","y":"663","bkcolor":"16777215","color":"0","code":"66","code_ascii":"B","confidence":"97","characater":"poly":"p":"x":"1164","y":"638","p":"x":"1173","y":"638","p":"x":"1173","y":"663","p":"x":"1163","y":"663","bkcolor":"16777215","color":"0","code":"66","code_ascii":"B","confidence":"94","characater":"poly":"p":"x":"1191","y":"637","p":"x":"1206","y":"636","p":"x":"1205","y":"660","p":"x":"1190","y":"661","bkcolor":"16777215","color":"0","code":"76","code_ascii":"L","confidence":"34","det_time_us":"1072592","poly":"p":"x":"1088","y":"642","p":"x":"1210","y":"634","p":"x":"1210","y":"661","p":"x":"1087","y":"669","det_time_us":"1720812"

link

input file: https://drive.google.com/file/d/18wCzjMBpw7SIeVFByAGPQiqCBjg_0te3/view?usp=sharing

Since your characters array is not a proper JSON array (it's a hash where each key has the same name), only the last value of it will be available.

We can fix that by preprocessing the data to create unique keys for each object that is now called characater:

awk '/"characater"/ sub(""characater"", ""char" ++n """, $0) 1' file.json

This would replace each characater with char1, char2 etc.

We can now access all values in that with e.g.

jq -r '.frames.frame.lps.lp|.characters|[.code_ascii,.confidence]|@tsv'

The complete pipeline:

awk '/"characater"/ sub(""characater"", ""char" ++n """, $0) 1' file.json |
jq -r '.frames.frame.lps.lp|.characters|[.code_ascii,.confidence]|@tsv'

Result (given the data in the question):

1 97
5 89
4 97
5 97
1 77
B 97
B 94
L 34

If you control the generation of the JSON document, you should consider turning the characters object into a proper array.

Given your input isn't valid JSON you'll likely have to go with a sed, awk, grep type of solution. To that end the following can deal with input where it's been 'minified' into a single string:

$ grep -oP '"code_ascii":"w+","confidence":"w+"' <FILE> | grep -oP '(?<=:)"w+"' | paste - -

Example

$ grep -oP '"code_ascii":"w+","confidence":"w+"' b.json | grep -oP '(?<=:)"w+"' | paste - -
"1", "97"
"5", "89"
"4", "97"
"5", "97"
"1", "77"
"B", "97"
"B", "94"
"L", "34"
"J", "57"
"J", "57"
"4", "7"
"1", "97"
"5", "89"
"4", "97"
"5", "97"
"1", "77"
"B", "97"
"B", "94"
"L", "34"

Alternatives

NOTE: These work with multi-line input data.

$ sed -n '/code_ascii/,/confidence/p' a.json | sed 's/.*: [^"]*//' | paste - -

$ sed -n '/code_ascii/,/confidence/p' a.json | awk -F': ' 'print $2' | paste -d" " - -

$ sed -n '/code_ascii/,/confidence/p' a.json | grep -oP '(?<=: ).*$' | paste - -

$ grep -oP '"code_ascii":"w+","confidence":"w+"' b.json | grep -oP '(?<=:)"w+"' | paste - -

grep + sed + tr :

grep -e code_ascii -e confidence <file> | sed 's/.*: "(.*)"/1/;' | tr "n" ' '

Return :

20, 1, 97 5, 89 4, 97 5, 97 1, 77 B, 97 B, 94 L, 34 J, 57 J, 57 4, 7 1, 97 5, 89 4, 97 5, 97 1, 77 B, 97 B, 94 L, 34 

This command return only the pairs code_ascii + confidence :

 grep -e code_ascii -e confidence <file> | sed -n 's/.*code_ascii": "(.*)"/1/;T;h;n;s/.*"confidence": "(.*)"/1/;T;H;g;p' | tr "n" " " 

Return :

 1, 97 5, 89 4, 97 5, 97 1, 77 B, 97 B, 94 L, 34 J, 57 J, 57 4, 7 1, 97 5, 89 4, 97 5, 97 1, 77 B, 97 B, 94 L, 34 

Now, see above the Kusalananda's post. With just sed + tr :

sed -i 's/},/},n/g' <tempfile>
sed -n 's/characater//;T;:z;n;s/.*code_ascii": "(.*)"/1/;Tz;:zz;h;n;s/.*"confidence": "(.*)"/1/;Tzz;H;s/$//;Tz;g;p' <tempfile> | tr "n" " "

Return :

1, 97 5, 89 4, 97 5, 97 1, 77 B, 97 B, 94 L, 34