Why is the JSON content from heredoc not parsable?
Clash Royale CLAN TAG#URR8PPP
up vote
10
down vote
favorite
I have a JSON fragment.
The following does not work:
VALUE=<<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
PERSON
echo -n "$VALUE" | python -m json.tool
The result is:
No JSON object could be decoded
Doing the same with jq
, i. e.
echo -n "$VALUE" | jq '.'
There is no output.
There is the same behavior for the following:
VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool
Response:
No JSON object could be decoded
But the following works:
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool
shell variable json here-document
add a comment |Â
up vote
10
down vote
favorite
I have a JSON fragment.
The following does not work:
VALUE=<<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
PERSON
echo -n "$VALUE" | python -m json.tool
The result is:
No JSON object could be decoded
Doing the same with jq
, i. e.
echo -n "$VALUE" | jq '.'
There is no output.
There is the same behavior for the following:
VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool
Response:
No JSON object could be decoded
But the following works:
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool
shell variable json here-document
5
I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
â Nick T
Apr 11 at 14:53
@NickT you should make that an answer as I think that is precisely the problem.
â rrauenza
Apr 11 at 18:20
If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simpleecho $VALUE
without... | jq
would be informative.
â Nick T
Apr 11 at 18:27
@NickT:That was a copy/paste issue. Sorry for the confusion
â Jim
Apr 11 at 21:00
add a comment |Â
up vote
10
down vote
favorite
up vote
10
down vote
favorite
I have a JSON fragment.
The following does not work:
VALUE=<<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
PERSON
echo -n "$VALUE" | python -m json.tool
The result is:
No JSON object could be decoded
Doing the same with jq
, i. e.
echo -n "$VALUE" | jq '.'
There is no output.
There is the same behavior for the following:
VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool
Response:
No JSON object could be decoded
But the following works:
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool
shell variable json here-document
I have a JSON fragment.
The following does not work:
VALUE=<<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
PERSON
echo -n "$VALUE" | python -m json.tool
The result is:
No JSON object could be decoded
Doing the same with jq
, i. e.
echo -n "$VALUE" | jq '.'
There is no output.
There is the same behavior for the following:
VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool
Response:
No JSON object could be decoded
But the following works:
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool
shell variable json here-document
edited Apr 11 at 21:02
asked Apr 11 at 10:59
Jim
39211
39211
5
I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
â Nick T
Apr 11 at 14:53
@NickT you should make that an answer as I think that is precisely the problem.
â rrauenza
Apr 11 at 18:20
If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simpleecho $VALUE
without... | jq
would be informative.
â Nick T
Apr 11 at 18:27
@NickT:That was a copy/paste issue. Sorry for the confusion
â Jim
Apr 11 at 21:00
add a comment |Â
5
I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
â Nick T
Apr 11 at 14:53
@NickT you should make that an answer as I think that is precisely the problem.
â rrauenza
Apr 11 at 18:20
If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simpleecho $VALUE
without... | jq
would be informative.
â Nick T
Apr 11 at 18:27
@NickT:That was a copy/paste issue. Sorry for the confusion
â Jim
Apr 11 at 21:00
5
5
I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
â Nick T
Apr 11 at 14:53
I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
â Nick T
Apr 11 at 14:53
@NickT you should make that an answer as I think that is precisely the problem.
â rrauenza
Apr 11 at 18:20
@NickT you should make that an answer as I think that is precisely the problem.
â rrauenza
Apr 11 at 18:20
If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple
echo $VALUE
without ... | jq
would be informative.â Nick T
Apr 11 at 18:27
If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple
echo $VALUE
without ... | jq
would be informative.â Nick T
Apr 11 at 18:27
@NickT:That was a copy/paste issue. Sorry for the confusion
â Jim
Apr 11 at 21:00
@NickT:That was a copy/paste issue. Sorry for the confusion
â Jim
Apr 11 at 21:00
add a comment |Â
4 Answers
4
active
oldest
votes
up vote
19
down vote
VALUE=<<PERSON
some data
PERSON
echo "$VALUE"
No output.
A here-document is a redirection, you can't redirect into a variable.
When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)
VALUE= <<PERSON
some data
PERSON
That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).
Note that
<<PERSON
some data
PERSON
is valid, as is
<somefile
It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.
This would work though:
VALUE=$(cat <<PERSON
some data
PERSON
)
Here, the command that receives the here-document is cat
, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.
In your case, you could instead use
python -m json.tool <<END_JSON
JSON data here
END_JSON
without taking the extra step of storing the data in a variable.
2
You could also just doPERSON="
followed by a newline and the multi-line data, then another"
at the end.
â R..
Apr 11 at 14:14
1
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
2
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.PERSON='
. That's unless the OP wants to interpolate variables later.
â JoL
Apr 11 at 15:39
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
 |Â
show 6 more comments
up vote
10
down vote
Because the variable isn't being set by your heredoc:
$ VALUE=<<PERSON
>
> "type": "account",
> "customer_id": "1234",
> "customer_email": "jim@gmail.com",
>
> PERSON
$ echo "$VALUE"
$
If you want to use a heredoc to assign a value to a variable, you need something like:
$ read -d '' -r VALUE <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
1
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest<<â¯PERSON
to protect against$
s in the input and backslashes at the ends of lines.
â Scott
Apr 11 at 18:17
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
3
This is the right answer.$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable viaread
, by comparison, is sane.
â Rich
Apr 11 at 19:18
I wouldnâÂÂt say that$(cat <<â¯EOF
⦠(data) â¦EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so isreadâ¯-dâ¯â¦â¯<<â¯EOF
â especiallyreadâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the$(cat <<â¯EOF
⦠(data) â¦EOF
)
fails if any lines end with(backslash) â see the comments under KusalanandaâÂÂs answer.
â Scott
Apr 12 at 1:53
add a comment |Â
up vote
4
down vote
It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as
VALUE=$(cat <<EOF
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
EOF
)
and doing printf "$VALUE"
should dump the JSON as expected.
1
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
add a comment |Â
up vote
3
down vote
Heredocs and variables don't mix well or at least not in this way. You can eitherâ¦
Pass the heredoc as the standard input of an application
python -m json.tool <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
orâ¦
Store multi-line text in a shell variable
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
'
I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:
VALUE="
"type": "account",
"customer_id": $ID,
"customer_email": "$EMAIL",
"
Then you can use the variable value later on.
echo -n "$VALUE" | python -m json.tool
add a comment |Â
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
19
down vote
VALUE=<<PERSON
some data
PERSON
echo "$VALUE"
No output.
A here-document is a redirection, you can't redirect into a variable.
When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)
VALUE= <<PERSON
some data
PERSON
That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).
Note that
<<PERSON
some data
PERSON
is valid, as is
<somefile
It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.
This would work though:
VALUE=$(cat <<PERSON
some data
PERSON
)
Here, the command that receives the here-document is cat
, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.
In your case, you could instead use
python -m json.tool <<END_JSON
JSON data here
END_JSON
without taking the extra step of storing the data in a variable.
2
You could also just doPERSON="
followed by a newline and the multi-line data, then another"
at the end.
â R..
Apr 11 at 14:14
1
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
2
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.PERSON='
. That's unless the OP wants to interpolate variables later.
â JoL
Apr 11 at 15:39
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
 |Â
show 6 more comments
up vote
19
down vote
VALUE=<<PERSON
some data
PERSON
echo "$VALUE"
No output.
A here-document is a redirection, you can't redirect into a variable.
When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)
VALUE= <<PERSON
some data
PERSON
That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).
Note that
<<PERSON
some data
PERSON
is valid, as is
<somefile
It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.
This would work though:
VALUE=$(cat <<PERSON
some data
PERSON
)
Here, the command that receives the here-document is cat
, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.
In your case, you could instead use
python -m json.tool <<END_JSON
JSON data here
END_JSON
without taking the extra step of storing the data in a variable.
2
You could also just doPERSON="
followed by a newline and the multi-line data, then another"
at the end.
â R..
Apr 11 at 14:14
1
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
2
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.PERSON='
. That's unless the OP wants to interpolate variables later.
â JoL
Apr 11 at 15:39
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
 |Â
show 6 more comments
up vote
19
down vote
up vote
19
down vote
VALUE=<<PERSON
some data
PERSON
echo "$VALUE"
No output.
A here-document is a redirection, you can't redirect into a variable.
When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)
VALUE= <<PERSON
some data
PERSON
That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).
Note that
<<PERSON
some data
PERSON
is valid, as is
<somefile
It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.
This would work though:
VALUE=$(cat <<PERSON
some data
PERSON
)
Here, the command that receives the here-document is cat
, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.
In your case, you could instead use
python -m json.tool <<END_JSON
JSON data here
END_JSON
without taking the extra step of storing the data in a variable.
VALUE=<<PERSON
some data
PERSON
echo "$VALUE"
No output.
A here-document is a redirection, you can't redirect into a variable.
When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)
VALUE= <<PERSON
some data
PERSON
That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).
Note that
<<PERSON
some data
PERSON
is valid, as is
<somefile
It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.
This would work though:
VALUE=$(cat <<PERSON
some data
PERSON
)
Here, the command that receives the here-document is cat
, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.
In your case, you could instead use
python -m json.tool <<END_JSON
JSON data here
END_JSON
without taking the extra step of storing the data in a variable.
edited Apr 12 at 6:40
answered Apr 11 at 11:05
Kusalananda
102k13199316
102k13199316
2
You could also just doPERSON="
followed by a newline and the multi-line data, then another"
at the end.
â R..
Apr 11 at 14:14
1
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
2
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.PERSON='
. That's unless the OP wants to interpolate variables later.
â JoL
Apr 11 at 15:39
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
 |Â
show 6 more comments
2
You could also just doPERSON="
followed by a newline and the multi-line data, then another"
at the end.
â R..
Apr 11 at 14:14
1
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
2
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.PERSON='
. That's unless the OP wants to interpolate variables later.
â JoL
Apr 11 at 15:39
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
2
2
You could also just do
PERSON="
followed by a newline and the multi-line data, then another "
at the end.â R..
Apr 11 at 14:14
You could also just do
PERSON="
followed by a newline and the multi-line data, then another "
at the end.â R..
Apr 11 at 14:14
1
1
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
@R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
â Kusalananda
Apr 11 at 14:20
2
2
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.
PERSON='
. That's unless the OP wants to interpolate variables later.â JoL
Apr 11 at 15:39
@R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name.
PERSON='
. That's unless the OP wants to interpolate variables later.â JoL
Apr 11 at 15:39
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
(backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word.âÂÂThat might be desirable, but is there any way to disable it?
â Scott
Apr 11 at 18:17
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
@Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
â Kusalananda
Apr 11 at 18:24
 |Â
show 6 more comments
up vote
10
down vote
Because the variable isn't being set by your heredoc:
$ VALUE=<<PERSON
>
> "type": "account",
> "customer_id": "1234",
> "customer_email": "jim@gmail.com",
>
> PERSON
$ echo "$VALUE"
$
If you want to use a heredoc to assign a value to a variable, you need something like:
$ read -d '' -r VALUE <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
1
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest<<â¯PERSON
to protect against$
s in the input and backslashes at the ends of lines.
â Scott
Apr 11 at 18:17
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
3
This is the right answer.$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable viaread
, by comparison, is sane.
â Rich
Apr 11 at 19:18
I wouldnâÂÂt say that$(cat <<â¯EOF
⦠(data) â¦EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so isreadâ¯-dâ¯â¦â¯<<â¯EOF
â especiallyreadâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the$(cat <<â¯EOF
⦠(data) â¦EOF
)
fails if any lines end with(backslash) â see the comments under KusalanandaâÂÂs answer.
â Scott
Apr 12 at 1:53
add a comment |Â
up vote
10
down vote
Because the variable isn't being set by your heredoc:
$ VALUE=<<PERSON
>
> "type": "account",
> "customer_id": "1234",
> "customer_email": "jim@gmail.com",
>
> PERSON
$ echo "$VALUE"
$
If you want to use a heredoc to assign a value to a variable, you need something like:
$ read -d '' -r VALUE <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
1
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest<<â¯PERSON
to protect against$
s in the input and backslashes at the ends of lines.
â Scott
Apr 11 at 18:17
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
3
This is the right answer.$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable viaread
, by comparison, is sane.
â Rich
Apr 11 at 19:18
I wouldnâÂÂt say that$(cat <<â¯EOF
⦠(data) â¦EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so isreadâ¯-dâ¯â¦â¯<<â¯EOF
â especiallyreadâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the$(cat <<â¯EOF
⦠(data) â¦EOF
)
fails if any lines end with(backslash) â see the comments under KusalanandaâÂÂs answer.
â Scott
Apr 12 at 1:53
add a comment |Â
up vote
10
down vote
up vote
10
down vote
Because the variable isn't being set by your heredoc:
$ VALUE=<<PERSON
>
> "type": "account",
> "customer_id": "1234",
> "customer_email": "jim@gmail.com",
>
> PERSON
$ echo "$VALUE"
$
If you want to use a heredoc to assign a value to a variable, you need something like:
$ read -d '' -r VALUE <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
Because the variable isn't being set by your heredoc:
$ VALUE=<<PERSON
>
> "type": "account",
> "customer_id": "1234",
> "customer_email": "jim@gmail.com",
>
> PERSON
$ echo "$VALUE"
$
If you want to use a heredoc to assign a value to a variable, you need something like:
$ read -d '' -r VALUE <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
edited Apr 11 at 18:28
answered Apr 11 at 11:05
terdonâ¦
122k28229400
122k28229400
1
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest<<â¯PERSON
to protect against$
s in the input and backslashes at the ends of lines.
â Scott
Apr 11 at 18:17
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
3
This is the right answer.$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable viaread
, by comparison, is sane.
â Rich
Apr 11 at 19:18
I wouldnâÂÂt say that$(cat <<â¯EOF
⦠(data) â¦EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so isreadâ¯-dâ¯â¦â¯<<â¯EOF
â especiallyreadâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the$(cat <<â¯EOF
⦠(data) â¦EOF
)
fails if any lines end with(backslash) â see the comments under KusalanandaâÂÂs answer.
â Scott
Apr 12 at 1:53
add a comment |Â
1
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest<<â¯PERSON
to protect against$
s in the input and backslashes at the ends of lines.
â Scott
Apr 11 at 18:17
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
3
This is the right answer.$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable viaread
, by comparison, is sane.
â Rich
Apr 11 at 19:18
I wouldnâÂÂt say that$(cat <<â¯EOF
⦠(data) â¦EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so isreadâ¯-dâ¯â¦â¯<<â¯EOF
â especiallyreadâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the$(cat <<â¯EOF
⦠(data) â¦EOF
)
fails if any lines end with(backslash) â see the comments under KusalanandaâÂÂs answer.
â Scott
Apr 12 at 1:53
1
1
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest
<<â¯PERSON
to protect against $
s in the input and backslashes at the ends of lines.â Scott
Apr 11 at 18:17
Why are you wrapping the JSON data in single quotes?â It doesnâÂÂt really look like the OP wants them to be part of his input string.â Aside from that,âÂÂ+1 for cutting down on the homeless cat population.â As with KusalanandaâÂÂs answer, you might want to suggest
<<â¯PERSON
to protect against $
s in the input and backslashes at the ends of lines.â Scott
Apr 11 at 18:17
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
@Scott um, because I just blindly copied the text from the OP. Thanks
â terdonâ¦
Apr 11 at 18:29
3
3
This is the right answer.
$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read
, by comparison, is sane.â Rich
Apr 11 at 19:18
This is the right answer.
$(cat <<EOF ... EOF)
is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read
, by comparison, is sane.â Rich
Apr 11 at 19:18
I wouldnâÂÂt say that
$(cat <<â¯EOF
⦠(data) ⦠EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so is readâ¯-dâ¯â¦â¯<<â¯EOF
â especially readâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the $(cat <<â¯EOF
⦠(data) ⦠EOF
)
fails if any lines end with
(backslash) â see the comments under KusalanandaâÂÂs answer.â Scott
Apr 12 at 1:53
I wouldnâÂÂt say that
$(cat <<â¯EOF
⦠(data) ⦠EOF
)
is weird.âÂÂItâÂÂs awkward and convoluted, but so is readâ¯-dâ¯â¦â¯<<â¯EOF
â especially readâ¯-dâ¯''â¯<<â¯EOF
.âÂÂI appreciate terdonâÂÂs answer because it uses only builtins, no programs.âÂÂBut, more importantly, the $(cat <<â¯EOF
⦠(data) ⦠EOF
)
fails if any lines end with
(backslash) â see the comments under KusalanandaâÂÂs answer.â Scott
Apr 12 at 1:53
add a comment |Â
up vote
4
down vote
It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as
VALUE=$(cat <<EOF
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
EOF
)
and doing printf "$VALUE"
should dump the JSON as expected.
1
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
add a comment |Â
up vote
4
down vote
It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as
VALUE=$(cat <<EOF
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
EOF
)
and doing printf "$VALUE"
should dump the JSON as expected.
1
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
add a comment |Â
up vote
4
down vote
up vote
4
down vote
It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as
VALUE=$(cat <<EOF
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
EOF
)
and doing printf "$VALUE"
should dump the JSON as expected.
It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as
VALUE=$(cat <<EOF
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
EOF
)
and doing printf "$VALUE"
should dump the JSON as expected.
edited Apr 11 at 11:11
answered Apr 11 at 11:05
Inian
2,855722
2,855722
1
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
add a comment |Â
1
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
1
1
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
Useless use of cat. Just put "" around it and don't use a heredoc.
â R..
Apr 11 at 14:15
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
â psmears
Apr 12 at 15:34
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
@psmears: Yes, I realized that after edit window closed. Should use single quotes.
â R..
Apr 12 at 15:54
add a comment |Â
up vote
3
down vote
Heredocs and variables don't mix well or at least not in this way. You can eitherâ¦
Pass the heredoc as the standard input of an application
python -m json.tool <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
orâ¦
Store multi-line text in a shell variable
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
'
I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:
VALUE="
"type": "account",
"customer_id": $ID,
"customer_email": "$EMAIL",
"
Then you can use the variable value later on.
echo -n "$VALUE" | python -m json.tool
add a comment |Â
up vote
3
down vote
Heredocs and variables don't mix well or at least not in this way. You can eitherâ¦
Pass the heredoc as the standard input of an application
python -m json.tool <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
orâ¦
Store multi-line text in a shell variable
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
'
I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:
VALUE="
"type": "account",
"customer_id": $ID,
"customer_email": "$EMAIL",
"
Then you can use the variable value later on.
echo -n "$VALUE" | python -m json.tool
add a comment |Â
up vote
3
down vote
up vote
3
down vote
Heredocs and variables don't mix well or at least not in this way. You can eitherâ¦
Pass the heredoc as the standard input of an application
python -m json.tool <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
orâ¦
Store multi-line text in a shell variable
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
'
I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:
VALUE="
"type": "account",
"customer_id": $ID,
"customer_email": "$EMAIL",
"
Then you can use the variable value later on.
echo -n "$VALUE" | python -m json.tool
Heredocs and variables don't mix well or at least not in this way. You can eitherâ¦
Pass the heredoc as the standard input of an application
python -m json.tool <<PERSON
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
PERSON
orâ¦
Store multi-line text in a shell variable
VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
'
I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:
VALUE="
"type": "account",
"customer_id": $ID,
"customer_email": "$EMAIL",
"
Then you can use the variable value later on.
echo -n "$VALUE" | python -m json.tool
answered Apr 11 at 12:10
David Foerster
915616
915616
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f436978%2fwhy-is-the-json-content-from-heredoc-not-parsable%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
5
I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
â Nick T
Apr 11 at 14:53
@NickT you should make that an answer as I think that is precisely the problem.
â rrauenza
Apr 11 at 18:20
If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple
echo $VALUE
without... | jq
would be informative.â Nick T
Apr 11 at 18:27
@NickT:That was a copy/paste issue. Sorry for the confusion
â Jim
Apr 11 at 21:00