Using jq to get a structured output
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I have the following json as my input for jq processing
[
"Category": "Disk Partition Details",
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Category": "Disk Partition Details",
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
using ./csvtojson.sh bb.csv | jq 'map( (.Category): del(.Category))' as suggested by @peak here, I've reached till the json below
[
"Disk Partition Details":
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Disk Partition Details":
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
All I want is to put the category on the top for once only and to break this json to another level as i did in the previous step like this.
[
"Disk Partition Details":
"udev" :
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs" :
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
I am new to linux, sorry if I am wrong, I will try my level best to describe my query if needed.
json jq
asked Mar 7 at 11:46
Ankit Sharma
83
add a comment |Â
up vote
1
down vote
favorite
I have the following json as my input for jq processing
[
"Category": "Disk Partition Details",
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Category": "Disk Partition Details",
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
using ./csvtojson.sh bb.csv | jq 'map( (.Category): del(.Category))' as suggested by @peak here, I've reached till the json below
[
"Disk Partition Details":
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Disk Partition Details":
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
All I want is to put the category on the top for once only and to break this json to another level as i did in the previous step like this.
[
"Disk Partition Details":
"udev" :
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs" :
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
I am new to linux, sorry if I am wrong, I will try my level best to describe my query if needed.
json jq
asked Mar 7 at 11:46
Ankit Sharma
83
1
I think it might be easier if you took the original CSV and used Python to create JSON output directly in the desired format.
– muru
Mar 7 at 11:54
Thanks, I agree this is easy in python. It's for learning how jq actually works.
– Ankit Sharma
Mar 7 at 11:57
The final JSON would be a bit difficult to work with. You would have to know to look in eitherudevortmpfsto get at theSize. It would be easier to query the JSON in the format it is at the top.
– Kusalananda
Mar 7 at 12:30
Thanks for pointing out @Kusalananda The json used above is just a sample here for understanting the usage of jq.
– Ankit Sharma
Mar 7 at 13:37
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have the following json as my input for jq processing
[
"Category": "Disk Partition Details",
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Category": "Disk Partition Details",
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
using ./csvtojson.sh bb.csv | jq 'map( (.Category): del(.Category))' as suggested by @peak here, I've reached till the json below
[
"Disk Partition Details":
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Disk Partition Details":
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
All I want is to put the category on the top for once only and to break this json to another level as i did in the previous step like this.
[
"Disk Partition Details":
"udev" :
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs" :
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
I am new to linux, sorry if I am wrong, I will try my level best to describe my query if needed.
json jq
asked Mar 7 at 11:46
Ankit Sharma
83
I have the following json as my input for jq processing
[
"Category": "Disk Partition Details",
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Category": "Disk Partition Details",
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
using ./csvtojson.sh bb.csv | jq 'map( (.Category): del(.Category))' as suggested by @peak here, I've reached till the json below
[
"Disk Partition Details":
"Filesystem": "udev",
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"Disk Partition Details":
"Filesystem": "tmpfs",
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
All I want is to put the category on the top for once only and to break this json to another level as i did in the previous step like this.
[
"Disk Partition Details":
"udev" :
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs" :
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
I am new to linux, sorry if I am wrong, I will try my level best to describe my query if needed.
json jq
asked Mar 7 at 11:46
Ankit Sharma
83
asked Mar 7 at 11:46
Ankit Sharma
83
asked Mar 7 at 11:46
Ankit Sharma
83
asked Mar 7 at 11:46
Ankit Sharma
83
83
1
I think it might be easier if you took the original CSV and used Python to create JSON output directly in the desired format.
– muru
Mar 7 at 11:54
Thanks, I agree this is easy in python. It's for learning how jq actually works.
– Ankit Sharma
Mar 7 at 11:57
The final JSON would be a bit difficult to work with. You would have to know to look in eitherudevortmpfsto get at theSize. It would be easier to query the JSON in the format it is at the top.
– Kusalananda
Mar 7 at 12:30
Thanks for pointing out @Kusalananda The json used above is just a sample here for understanting the usage of jq.
– Ankit Sharma
Mar 7 at 13:37
add a comment |Â
1
I think it might be easier if you took the original CSV and used Python to create JSON output directly in the desired format.
– muru
Mar 7 at 11:54
Thanks, I agree this is easy in python. It's for learning how jq actually works.
– Ankit Sharma
Mar 7 at 11:57
The final JSON would be a bit difficult to work with. You would have to know to look in eitherudevortmpfsto get at theSize. It would be easier to query the JSON in the format it is at the top.
– Kusalananda
Mar 7 at 12:30
Thanks for pointing out @Kusalananda The json used above is just a sample here for understanting the usage of jq.
– Ankit Sharma
Mar 7 at 13:37
1
1
I think it might be easier if you took the original CSV and used Python to create JSON output directly in the desired format.
– muru
Mar 7 at 11:54
I think it might be easier if you took the original CSV and used Python to create JSON output directly in the desired format.
– muru
Mar 7 at 11:54
Thanks, I agree this is easy in python. It's for learning how jq actually works.
– Ankit Sharma
Mar 7 at 11:57
Thanks, I agree this is easy in python. It's for learning how jq actually works.
– Ankit Sharma
Mar 7 at 11:57
The final JSON would be a bit difficult to work with. You would have to know to look in either
udev or tmpfs to get at the Size. It would be easier to query the JSON in the format it is at the top.– Kusalananda
Mar 7 at 12:30
The final JSON would be a bit difficult to work with. You would have to know to look in either
udev or tmpfs to get at the Size. It would be easier to query the JSON in the format it is at the top.– Kusalananda
Mar 7 at 12:30
Thanks for pointing out @Kusalananda The json used above is just a sample here for understanting the usage of jq.
– Ankit Sharma
Mar 7 at 13:37
Thanks for pointing out @Kusalananda The json used above is just a sample here for understanting the usage of jq.
– Ankit Sharma
Mar 7 at 13:37
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
1
down vote
accepted
Complete jq solution:
jq '[ group_by(.Category)[0] | .[0].Category as $k
| del($o.Category, $o.Filesystem)))
)
]' input.json
In case if "Filesystem" key could be a number - change .[($o.Filesystem)] to the following .[($o.Filesystem | tostring)]
The output:
[
"Disk Partition Details":
"udev":
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs":
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
@AnkitSharma, read aboutgroup_by()function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)
– RomanPerekhrest
Mar 7 at 13:38
add a comment |Â
up vote
1
down vote
You are doing yourself a disservice rearranging your JSON in this way.
The original JSON looks fine to me.
With that, if you'd like to get all the sizes for udev filesystems you would do
jq '. | select(.Filesystem == "udev").Size' file.json
A simple operation like that on the reorganized file would be
jq '. | ."Disk Partition Details".udev.Size' file1.json
The rearranged file furthermore only supports one mountpoint per filesystem type as far as I can see. If you had several tmpfs filesystems, then I don't know where you would put them.
answered Mar 7 at 12:42
Kusalananda
103k13202318
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
Complete jq solution:
jq '[ group_by(.Category)[0] | .[0].Category as $k
| del($o.Category, $o.Filesystem)))
)
]' input.json
In case if "Filesystem" key could be a number - change .[($o.Filesystem)] to the following .[($o.Filesystem | tostring)]
The output:
[
"Disk Partition Details":
"udev":
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs":
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
@AnkitSharma, read aboutgroup_by()function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)
– RomanPerekhrest
Mar 7 at 13:38
add a comment |Â
up vote
1
down vote
accepted
Complete jq solution:
jq '[ group_by(.Category)[0] | .[0].Category as $k
| del($o.Category, $o.Filesystem)))
)
]' input.json
In case if "Filesystem" key could be a number - change .[($o.Filesystem)] to the following .[($o.Filesystem | tostring)]
The output:
[
"Disk Partition Details":
"udev":
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs":
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
@AnkitSharma, read aboutgroup_by()function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)
– RomanPerekhrest
Mar 7 at 13:38
add a comment |Â
up vote
1
down vote
accepted
up vote
1
down vote
accepted
Complete jq solution:
jq '[ group_by(.Category)[0] | .[0].Category as $k
| del($o.Category, $o.Filesystem)))
)
]' input.json
In case if "Filesystem" key could be a number - change .[($o.Filesystem)] to the following .[($o.Filesystem | tostring)]
The output:
[
"Disk Partition Details":
"udev":
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs":
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
Complete jq solution:
jq '[ group_by(.Category)[0] | .[0].Category as $k
| del($o.Category, $o.Filesystem)))
)
]' input.json
In case if "Filesystem" key could be a number - change .[($o.Filesystem)] to the following .[($o.Filesystem | tostring)]
The output:
[
"Disk Partition Details":
"udev":
"Size": "3.9G",
"Used": 0,
"Avail": "3.9G",
"Use%": "0%",
"Mounted": "/dev"
,
"tmpfs":
"Size": "799M",
"Used": "34M",
"Avail": "766M",
"Use%": "5%",
"Mounted": "/run"
]
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
edited Mar 7 at 13:21
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
answered Mar 7 at 12:35
RomanPerekhrest
22.4k12144
22.4k12144
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
@AnkitSharma, read aboutgroup_by()function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)
– RomanPerekhrest
Mar 7 at 13:38
add a comment |Â
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
@AnkitSharma, read aboutgroup_by()function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)
– RomanPerekhrest
Mar 7 at 13:38
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
Thanks a lot, It is working perfectly fine. But it won't work if a value in Filesystem column is a number. It is throwing an error as jq: error (at <stdin>:83): Cannot index object with number.
– Ankit Sharma
Mar 7 at 12:50
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
@AnkitSharma, see my notation
– RomanPerekhrest
Mar 7 at 13:06
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
Thanks a lot, now the numeric values are also working fine. But why it is giving me unique rows only? It would be a great help if you would explain it to me.
– Ankit Sharma
Mar 7 at 13:35
@AnkitSharma, read about
group_by() function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)– RomanPerekhrest
Mar 7 at 13:38
@AnkitSharma, read about
group_by() function: stedolan.github.io/jq/manual/v1.5/#group_by(path_expression)– RomanPerekhrest
Mar 7 at 13:38
add a comment |Â
up vote
1
down vote
You are doing yourself a disservice rearranging your JSON in this way.
The original JSON looks fine to me.
With that, if you'd like to get all the sizes for udev filesystems you would do
jq '. | select(.Filesystem == "udev").Size' file.json
A simple operation like that on the reorganized file would be
jq '. | ."Disk Partition Details".udev.Size' file1.json
The rearranged file furthermore only supports one mountpoint per filesystem type as far as I can see. If you had several tmpfs filesystems, then I don't know where you would put them.
answered Mar 7 at 12:42
Kusalananda
103k13202318
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
add a comment |Â
up vote
1
down vote
You are doing yourself a disservice rearranging your JSON in this way.
The original JSON looks fine to me.
With that, if you'd like to get all the sizes for udev filesystems you would do
jq '. | select(.Filesystem == "udev").Size' file.json
A simple operation like that on the reorganized file would be
jq '. | ."Disk Partition Details".udev.Size' file1.json
The rearranged file furthermore only supports one mountpoint per filesystem type as far as I can see. If you had several tmpfs filesystems, then I don't know where you would put them.
answered Mar 7 at 12:42
Kusalananda
103k13202318
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
add a comment |Â
up vote
1
down vote
up vote
1
down vote
You are doing yourself a disservice rearranging your JSON in this way.
The original JSON looks fine to me.
With that, if you'd like to get all the sizes for udev filesystems you would do
jq '. | select(.Filesystem == "udev").Size' file.json
A simple operation like that on the reorganized file would be
jq '. | ."Disk Partition Details".udev.Size' file1.json
The rearranged file furthermore only supports one mountpoint per filesystem type as far as I can see. If you had several tmpfs filesystems, then I don't know where you would put them.
answered Mar 7 at 12:42
Kusalananda
103k13202318
You are doing yourself a disservice rearranging your JSON in this way.
The original JSON looks fine to me.
With that, if you'd like to get all the sizes for udev filesystems you would do
jq '. | select(.Filesystem == "udev").Size' file.json
A simple operation like that on the reorganized file would be
jq '. | ."Disk Partition Details".udev.Size' file1.json
The rearranged file furthermore only supports one mountpoint per filesystem type as far as I can see. If you had several tmpfs filesystems, then I don't know where you would put them.
answered Mar 7 at 12:42
Kusalananda
103k13202318
answered Mar 7 at 12:42
Kusalananda
103k13202318
answered Mar 7 at 12:42
Kusalananda
103k13202318
answered Mar 7 at 12:42
Kusalananda
103k13202318
103k13202318
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
add a comment |Â
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
As mentioned above, The json used above is just a sample here for understating the usage of jq. Roman answer serves it very well. Thanks for your comments though.
– Ankit Sharma
Mar 9 at 12:57
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f428727%2fusing-jq-to-get-a-structured-output%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
I think it might be easier if you took the original CSV and used Python to create JSON output directly in the desired format.
– muru
Mar 7 at 11:54
Thanks, I agree this is easy in python. It's for learning how jq actually works.
– Ankit Sharma
Mar 7 at 11:57
The final JSON would be a bit difficult to work with. You would have to know to look in either
udevortmpfsto get at theSize. It would be easier to query the JSON in the format it is at the top.– Kusalananda
Mar 7 at 12:30
Thanks for pointing out @Kusalananda The json used above is just a sample here for understanting the usage of jq.
– Ankit Sharma
Mar 7 at 13:37