Why is the JSON content from heredoc not parsable?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
10
down vote

favorite












I have a JSON fragment.



The following does not work:





VALUE=<<PERSON

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"

PERSON
echo -n "$VALUE" | python -m json.tool


The result is:




No JSON object could be decoded




Doing the same with jq, i. e.



echo -n "$VALUE" | jq '.'


There is no output.



There is the same behavior for the following:



VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool


Response:




No JSON object could be decoded




But the following works:



VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool






share|improve this question


















  • 5




    I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
    – Nick T
    Apr 11 at 14:53











  • @NickT you should make that an answer as I think that is precisely the problem.
    – rrauenza
    Apr 11 at 18:20










  • If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple echo $VALUE without ... | jq would be informative.
    – Nick T
    Apr 11 at 18:27










  • @NickT:That was a copy/paste issue. Sorry for the confusion
    – Jim
    Apr 11 at 21:00














up vote
10
down vote

favorite












I have a JSON fragment.



The following does not work:





VALUE=<<PERSON

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"

PERSON
echo -n "$VALUE" | python -m json.tool


The result is:




No JSON object could be decoded




Doing the same with jq, i. e.



echo -n "$VALUE" | jq '.'


There is no output.



There is the same behavior for the following:



VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool


Response:




No JSON object could be decoded




But the following works:



VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool






share|improve this question


















  • 5




    I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
    – Nick T
    Apr 11 at 14:53











  • @NickT you should make that an answer as I think that is precisely the problem.
    – rrauenza
    Apr 11 at 18:20










  • If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple echo $VALUE without ... | jq would be informative.
    – Nick T
    Apr 11 at 18:27










  • @NickT:That was a copy/paste issue. Sorry for the confusion
    – Jim
    Apr 11 at 21:00












up vote
10
down vote

favorite









up vote
10
down vote

favorite











I have a JSON fragment.



The following does not work:





VALUE=<<PERSON

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"

PERSON
echo -n "$VALUE" | python -m json.tool


The result is:




No JSON object could be decoded




Doing the same with jq, i. e.



echo -n "$VALUE" | jq '.'


There is no output.



There is the same behavior for the following:



VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool


Response:




No JSON object could be decoded




But the following works:



VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool






share|improve this question














I have a JSON fragment.



The following does not work:





VALUE=<<PERSON

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"

PERSON
echo -n "$VALUE" | python -m json.tool


The result is:




No JSON object could be decoded




Doing the same with jq, i. e.



echo -n "$VALUE" | jq '.'


There is no output.



There is the same behavior for the following:



VALUE=<<PERSON
'
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
PERSON
echo -n "$VALUE" | python -m json.tool


Response:




No JSON object could be decoded




But the following works:



VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com"
'
echo -n "$VALUE" | jq '.'
echo -n "$VALUE" | python -m json.tool








share|improve this question













share|improve this question




share|improve this question








edited Apr 11 at 21:02

























asked Apr 11 at 10:59









Jim

39211




39211







  • 5




    I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
    – Nick T
    Apr 11 at 14:53











  • @NickT you should make that an answer as I think that is precisely the problem.
    – rrauenza
    Apr 11 at 18:20










  • If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple echo $VALUE without ... | jq would be informative.
    – Nick T
    Apr 11 at 18:27










  • @NickT:That was a copy/paste issue. Sorry for the confusion
    – Jim
    Apr 11 at 21:00












  • 5




    I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
    – Nick T
    Apr 11 at 14:53











  • @NickT you should make that an answer as I think that is precisely the problem.
    – rrauenza
    Apr 11 at 18:20










  • If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple echo $VALUE without ... | jq would be informative.
    – Nick T
    Apr 11 at 18:27










  • @NickT:That was a copy/paste issue. Sorry for the confusion
    – Jim
    Apr 11 at 21:00







5




5




I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
– Nick T
Apr 11 at 14:53





I don't know what bash is doing, but there's a trailing comma after the email string in your first two but not on the third, which would make the first couple illegal JSON
– Nick T
Apr 11 at 14:53













@NickT you should make that an answer as I think that is precisely the problem.
– rrauenza
Apr 11 at 18:20




@NickT you should make that an answer as I think that is precisely the problem.
– rrauenza
Apr 11 at 18:20












If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple echo $VALUE without ... | jq would be informative.
– Nick T
Apr 11 at 18:27




If that's the (sole) answer it should probably be closed as "can't be reproduced (a typo)". However, it looks like Kusa's and terdon's answer mention the assignment + redirection is totally broken so you get an empty string, so there are two problems, both of which would give the same "No JSON..." error. It's very good practice to bisect problems by checking your assumptions in the middle: a simple echo $VALUE without ... | jq would be informative.
– Nick T
Apr 11 at 18:27












@NickT:That was a copy/paste issue. Sorry for the confusion
– Jim
Apr 11 at 21:00




@NickT:That was a copy/paste issue. Sorry for the confusion
– Jim
Apr 11 at 21:00










4 Answers
4






active

oldest

votes

















up vote
19
down vote













VALUE=<<PERSON
some data
PERSON

echo "$VALUE"


No output.



A here-document is a redirection, you can't redirect into a variable.



When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)



VALUE= <<PERSON
some data
PERSON


That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).



Note that



<<PERSON
some data
PERSON


is valid, as is



<somefile


It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.



This would work though:



VALUE=$(cat <<PERSON
some data
PERSON
)


Here, the command that receives the here-document is cat, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.



In your case, you could instead use



python -m json.tool <<END_JSON
JSON data here
END_JSON


without taking the extra step of storing the data in a variable.






share|improve this answer


















  • 2




    You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
    – R..
    Apr 11 at 14:14






  • 1




    @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
    – Kusalananda
    Apr 11 at 14:20







  • 2




    @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
    – JoL
    Apr 11 at 15:39










  • (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
    – Scott
    Apr 11 at 18:17










  • @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
    – Kusalananda
    Apr 11 at 18:24

















up vote
10
down vote













Because the variable isn't being set by your heredoc:



$ VALUE=<<PERSON 
>
> "type": "account",
> "customer_id": "1234",
> "customer_email": "jim@gmail.com",
>
> PERSON
$ echo "$VALUE"

$


If you want to use a heredoc to assign a value to a variable, you need something like:



$ read -d '' -r VALUE <<PERSON 

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",

PERSON





share|improve this answer


















  • 1




    Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
    – Scott
    Apr 11 at 18:17










  • @Scott um, because I just blindly copied the text from the OP. Thanks
    – terdon♦
    Apr 11 at 18:29






  • 3




    This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
    – Rich
    Apr 11 at 19:18










  • I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
    – Scott
    Apr 12 at 1:53

















up vote
4
down vote













It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as



VALUE=$(cat <<EOF

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",

EOF
)


and doing printf "$VALUE" should dump the JSON as expected.






share|improve this answer


















  • 1




    Useless use of cat. Just put "" around it and don't use a heredoc.
    – R..
    Apr 11 at 14:15










  • @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
    – psmears
    Apr 12 at 15:34










  • @psmears: Yes, I realized that after edit window closed. Should use single quotes.
    – R..
    Apr 12 at 15:54

















up vote
3
down vote













Heredocs and variables don't mix well or at least not in this way. You can either…



Pass the heredoc as the standard input of an application





python -m json.tool <<PERSON 

"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",

PERSON


or…



Store multi-line text in a shell variable



VALUE='
"type": "account",
"customer_id": "1234",
"customer_email": "jim@gmail.com",
'


I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:



VALUE="
"type": "account",
"customer_id": $ID,
"customer_email": "$EMAIL",
"


Then you can use the variable value later on.



echo -n "$VALUE" | python -m json.tool





share|improve this answer




















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );








     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f436978%2fwhy-is-the-json-content-from-heredoc-not-parsable%23new-answer', 'question_page');

    );

    Post as a guest






























    4 Answers
    4






    active

    oldest

    votes








    4 Answers
    4






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    19
    down vote













    VALUE=<<PERSON
    some data
    PERSON

    echo "$VALUE"


    No output.



    A here-document is a redirection, you can't redirect into a variable.



    When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)



    VALUE= <<PERSON
    some data
    PERSON


    That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).



    Note that



    <<PERSON
    some data
    PERSON


    is valid, as is



    <somefile


    It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.



    This would work though:



    VALUE=$(cat <<PERSON
    some data
    PERSON
    )


    Here, the command that receives the here-document is cat, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.



    In your case, you could instead use



    python -m json.tool <<END_JSON
    JSON data here
    END_JSON


    without taking the extra step of storing the data in a variable.






    share|improve this answer


















    • 2




      You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
      – R..
      Apr 11 at 14:14






    • 1




      @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
      – Kusalananda
      Apr 11 at 14:20







    • 2




      @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
      – JoL
      Apr 11 at 15:39










    • (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
      – Scott
      Apr 11 at 18:17










    • @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
      – Kusalananda
      Apr 11 at 18:24














    up vote
    19
    down vote













    VALUE=<<PERSON
    some data
    PERSON

    echo "$VALUE"


    No output.



    A here-document is a redirection, you can't redirect into a variable.



    When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)



    VALUE= <<PERSON
    some data
    PERSON


    That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).



    Note that



    <<PERSON
    some data
    PERSON


    is valid, as is



    <somefile


    It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.



    This would work though:



    VALUE=$(cat <<PERSON
    some data
    PERSON
    )


    Here, the command that receives the here-document is cat, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.



    In your case, you could instead use



    python -m json.tool <<END_JSON
    JSON data here
    END_JSON


    without taking the extra step of storing the data in a variable.






    share|improve this answer


















    • 2




      You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
      – R..
      Apr 11 at 14:14






    • 1




      @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
      – Kusalananda
      Apr 11 at 14:20







    • 2




      @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
      – JoL
      Apr 11 at 15:39










    • (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
      – Scott
      Apr 11 at 18:17










    • @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
      – Kusalananda
      Apr 11 at 18:24












    up vote
    19
    down vote










    up vote
    19
    down vote









    VALUE=<<PERSON
    some data
    PERSON

    echo "$VALUE"


    No output.



    A here-document is a redirection, you can't redirect into a variable.



    When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)



    VALUE= <<PERSON
    some data
    PERSON


    That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).



    Note that



    <<PERSON
    some data
    PERSON


    is valid, as is



    <somefile


    It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.



    This would work though:



    VALUE=$(cat <<PERSON
    some data
    PERSON
    )


    Here, the command that receives the here-document is cat, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.



    In your case, you could instead use



    python -m json.tool <<END_JSON
    JSON data here
    END_JSON


    without taking the extra step of storing the data in a variable.






    share|improve this answer














    VALUE=<<PERSON
    some data
    PERSON

    echo "$VALUE"


    No output.



    A here-document is a redirection, you can't redirect into a variable.



    When the command line is parsed, redirections are handled in a separate step from variable assignments. Your command is therefore equivalent to (note the space)



    VALUE= <<PERSON
    some data
    PERSON


    That is, it assigns an empty string to your variable, then redirects standard input from the here-string into the command (but there is no command, so nothing happens).



    Note that



    <<PERSON
    some data
    PERSON


    is valid, as is



    <somefile


    It's just that there is no command whose standard input stream can be set to contain the data, so it's just lost.



    This would work though:



    VALUE=$(cat <<PERSON
    some data
    PERSON
    )


    Here, the command that receives the here-document is cat, and it copies it to its standard output. This is then what is assigned to the variable by means of the command substitution.



    In your case, you could instead use



    python -m json.tool <<END_JSON
    JSON data here
    END_JSON


    without taking the extra step of storing the data in a variable.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Apr 12 at 6:40

























    answered Apr 11 at 11:05









    Kusalananda

    102k13199316




    102k13199316







    • 2




      You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
      – R..
      Apr 11 at 14:14






    • 1




      @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
      – Kusalananda
      Apr 11 at 14:20







    • 2




      @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
      – JoL
      Apr 11 at 15:39










    • (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
      – Scott
      Apr 11 at 18:17










    • @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
      – Kusalananda
      Apr 11 at 18:24












    • 2




      You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
      – R..
      Apr 11 at 14:14






    • 1




      @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
      – Kusalananda
      Apr 11 at 14:20







    • 2




      @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
      – JoL
      Apr 11 at 15:39










    • (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
      – Scott
      Apr 11 at 18:17










    • @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
      – Kusalananda
      Apr 11 at 18:24







    2




    2




    You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
    – R..
    Apr 11 at 14:14




    You could also just do PERSON=" followed by a newline and the multi-line data, then another " at the end.
    – R..
    Apr 11 at 14:14




    1




    1




    @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
    – Kusalananda
    Apr 11 at 14:20





    @R.. Yes, but a here-document allows you to bypass the quoting rules of the shell. It is therefore often safer to use a here-document instead of a quoted string for multi-line data, especially if the data contains single or double quotes (or both).
    – Kusalananda
    Apr 11 at 14:20





    2




    2




    @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
    – JoL
    Apr 11 at 15:39




    @R.. Given it's JSON we're talking about, it might be better to use single quotes to not have to escape the double quotes of each property name. PERSON='. That's unless the OP wants to interpolate variables later.
    – JoL
    Apr 11 at 15:39












    (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
    – Scott
    Apr 11 at 18:17




    (backslash)(newline) seems to vanish in a here document, even if you quote/escape the delimiter word. That might be desirable, but is there any way to disable it?
    – Scott
    Apr 11 at 18:17












    @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
    – Kusalananda
    Apr 11 at 18:24




    @Scott If that question hasn't been asked on this site before, it would be an excellent question in its own right.
    – Kusalananda
    Apr 11 at 18:24












    up vote
    10
    down vote













    Because the variable isn't being set by your heredoc:



    $ VALUE=<<PERSON 
    >
    > "type": "account",
    > "customer_id": "1234",
    > "customer_email": "jim@gmail.com",
    >
    > PERSON
    $ echo "$VALUE"

    $


    If you want to use a heredoc to assign a value to a variable, you need something like:



    $ read -d '' -r VALUE <<PERSON 

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    PERSON





    share|improve this answer


















    • 1




      Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
      – Scott
      Apr 11 at 18:17










    • @Scott um, because I just blindly copied the text from the OP. Thanks
      – terdon♦
      Apr 11 at 18:29






    • 3




      This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
      – Rich
      Apr 11 at 19:18










    • I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
      – Scott
      Apr 12 at 1:53














    up vote
    10
    down vote













    Because the variable isn't being set by your heredoc:



    $ VALUE=<<PERSON 
    >
    > "type": "account",
    > "customer_id": "1234",
    > "customer_email": "jim@gmail.com",
    >
    > PERSON
    $ echo "$VALUE"

    $


    If you want to use a heredoc to assign a value to a variable, you need something like:



    $ read -d '' -r VALUE <<PERSON 

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    PERSON





    share|improve this answer


















    • 1




      Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
      – Scott
      Apr 11 at 18:17










    • @Scott um, because I just blindly copied the text from the OP. Thanks
      – terdon♦
      Apr 11 at 18:29






    • 3




      This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
      – Rich
      Apr 11 at 19:18










    • I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
      – Scott
      Apr 12 at 1:53












    up vote
    10
    down vote










    up vote
    10
    down vote









    Because the variable isn't being set by your heredoc:



    $ VALUE=<<PERSON 
    >
    > "type": "account",
    > "customer_id": "1234",
    > "customer_email": "jim@gmail.com",
    >
    > PERSON
    $ echo "$VALUE"

    $


    If you want to use a heredoc to assign a value to a variable, you need something like:



    $ read -d '' -r VALUE <<PERSON 

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    PERSON





    share|improve this answer














    Because the variable isn't being set by your heredoc:



    $ VALUE=<<PERSON 
    >
    > "type": "account",
    > "customer_id": "1234",
    > "customer_email": "jim@gmail.com",
    >
    > PERSON
    $ echo "$VALUE"

    $


    If you want to use a heredoc to assign a value to a variable, you need something like:



    $ read -d '' -r VALUE <<PERSON 

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    PERSON






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Apr 11 at 18:28

























    answered Apr 11 at 11:05









    terdon♦

    122k28229400




    122k28229400







    • 1




      Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
      – Scott
      Apr 11 at 18:17










    • @Scott um, because I just blindly copied the text from the OP. Thanks
      – terdon♦
      Apr 11 at 18:29






    • 3




      This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
      – Rich
      Apr 11 at 19:18










    • I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
      – Scott
      Apr 12 at 1:53












    • 1




      Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
      – Scott
      Apr 11 at 18:17










    • @Scott um, because I just blindly copied the text from the OP. Thanks
      – terdon♦
      Apr 11 at 18:29






    • 3




      This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
      – Rich
      Apr 11 at 19:18










    • I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
      – Scott
      Apr 12 at 1:53







    1




    1




    Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
    – Scott
    Apr 11 at 18:17




    Why are you wrapping the JSON data in single quotes?  It doesn’t really look like the OP wants them to be part of his input string.  Aside from that, +1 for cutting down on the homeless cat population.  As with Kusalananda’s answer, you might want to suggest << PERSON to protect against $s in the input and backslashes at the ends of lines.
    – Scott
    Apr 11 at 18:17












    @Scott um, because I just blindly copied the text from the OP. Thanks
    – terdon♦
    Apr 11 at 18:29




    @Scott um, because I just blindly copied the text from the OP. Thanks
    – terdon♦
    Apr 11 at 18:29




    3




    3




    This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
    – Rich
    Apr 11 at 19:18




    This is the right answer. $(cat <<EOF ... EOF) is a weird construct: running a subshell and then sending a heredoc to cat just so it sends it to STDOUT and then assigning the result of that subshell to a variable? I wish people would think about what they're saying about their thought processes. Assigning a heredoc to a variable via read, by comparison, is sane.
    – Rich
    Apr 11 at 19:18












    I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
    – Scott
    Apr 12 at 1:53




    I wouldn’t say that $(cat << EOF … (data) … EOF ) is weird. It’s awkward and convoluted, but so is read -d … << EOF — especially read -d '' << EOF . I appreciate terdon’s answer because it uses only builtins, no programs. But, more importantly, the $(cat << EOF … (data) … EOF ) fails if any lines end with (backslash) — see the comments under Kusalananda’s answer.
    – Scott
    Apr 12 at 1:53










    up vote
    4
    down vote













    It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as



    VALUE=$(cat <<EOF

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    EOF
    )


    and doing printf "$VALUE" should dump the JSON as expected.






    share|improve this answer


















    • 1




      Useless use of cat. Just put "" around it and don't use a heredoc.
      – R..
      Apr 11 at 14:15










    • @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
      – psmears
      Apr 12 at 15:34










    • @psmears: Yes, I realized that after edit window closed. Should use single quotes.
      – R..
      Apr 12 at 15:54














    up vote
    4
    down vote













    It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as



    VALUE=$(cat <<EOF

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    EOF
    )


    and doing printf "$VALUE" should dump the JSON as expected.






    share|improve this answer


















    • 1




      Useless use of cat. Just put "" around it and don't use a heredoc.
      – R..
      Apr 11 at 14:15










    • @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
      – psmears
      Apr 12 at 15:34










    • @psmears: Yes, I realized that after edit window closed. Should use single quotes.
      – R..
      Apr 12 at 15:54












    up vote
    4
    down vote










    up vote
    4
    down vote









    It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as



    VALUE=$(cat <<EOF

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    EOF
    )


    and doing printf "$VALUE" should dump the JSON as expected.






    share|improve this answer














    It is because the way you have defined a here-doc to use with a JSON is wrong. You need to use it as



    VALUE=$(cat <<EOF

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    EOF
    )


    and doing printf "$VALUE" should dump the JSON as expected.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Apr 11 at 11:11

























    answered Apr 11 at 11:05









    Inian

    2,855722




    2,855722







    • 1




      Useless use of cat. Just put "" around it and don't use a heredoc.
      – R..
      Apr 11 at 14:15










    • @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
      – psmears
      Apr 12 at 15:34










    • @psmears: Yes, I realized that after edit window closed. Should use single quotes.
      – R..
      Apr 12 at 15:54












    • 1




      Useless use of cat. Just put "" around it and don't use a heredoc.
      – R..
      Apr 11 at 14:15










    • @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
      – psmears
      Apr 12 at 15:34










    • @psmears: Yes, I realized that after edit window closed. Should use single quotes.
      – R..
      Apr 12 at 15:54







    1




    1




    Useless use of cat. Just put "" around it and don't use a heredoc.
    – R..
    Apr 11 at 14:15




    Useless use of cat. Just put "" around it and don't use a heredoc.
    – R..
    Apr 11 at 14:15












    @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
    – psmears
    Apr 12 at 15:34




    @R..: Putting double quotes around it isn't that helpful a suggestion as the data contains double quotes already - did you mean single quotes?
    – psmears
    Apr 12 at 15:34












    @psmears: Yes, I realized that after edit window closed. Should use single quotes.
    – R..
    Apr 12 at 15:54




    @psmears: Yes, I realized that after edit window closed. Should use single quotes.
    – R..
    Apr 12 at 15:54










    up vote
    3
    down vote













    Heredocs and variables don't mix well or at least not in this way. You can either…



    Pass the heredoc as the standard input of an application





    python -m json.tool <<PERSON 

    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",

    PERSON


    or…



    Store multi-line text in a shell variable



    VALUE='
    "type": "account",
    "customer_id": "1234",
    "customer_email": "jim@gmail.com",
    '


    I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:



    VALUE="
    "type": "account",
    "customer_id": $ID,
    "customer_email": "$EMAIL",
    "


    Then you can use the variable value later on.



    echo -n "$VALUE" | python -m json.tool





    share|improve this answer
























      up vote
      3
      down vote













      Heredocs and variables don't mix well or at least not in this way. You can either…



      Pass the heredoc as the standard input of an application





      python -m json.tool <<PERSON 

      "type": "account",
      "customer_id": "1234",
      "customer_email": "jim@gmail.com",

      PERSON


      or…



      Store multi-line text in a shell variable



      VALUE='
      "type": "account",
      "customer_id": "1234",
      "customer_email": "jim@gmail.com",
      '


      I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:



      VALUE="
      "type": "account",
      "customer_id": $ID,
      "customer_email": "$EMAIL",
      "


      Then you can use the variable value later on.



      echo -n "$VALUE" | python -m json.tool





      share|improve this answer






















        up vote
        3
        down vote










        up vote
        3
        down vote









        Heredocs and variables don't mix well or at least not in this way. You can either…



        Pass the heredoc as the standard input of an application





        python -m json.tool <<PERSON 

        "type": "account",
        "customer_id": "1234",
        "customer_email": "jim@gmail.com",

        PERSON


        or…



        Store multi-line text in a shell variable



        VALUE='
        "type": "account",
        "customer_id": "1234",
        "customer_email": "jim@gmail.com",
        '


        I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:



        VALUE="
        "type": "account",
        "customer_id": $ID,
        "customer_email": "$EMAIL",
        "


        Then you can use the variable value later on.



        echo -n "$VALUE" | python -m json.tool





        share|improve this answer












        Heredocs and variables don't mix well or at least not in this way. You can either…



        Pass the heredoc as the standard input of an application





        python -m json.tool <<PERSON 

        "type": "account",
        "customer_id": "1234",
        "customer_email": "jim@gmail.com",

        PERSON


        or…



        Store multi-line text in a shell variable



        VALUE='
        "type": "account",
        "customer_id": "1234",
        "customer_email": "jim@gmail.com",
        '


        I used single quotes to avoid the need to escape the inner double quotes. Of course you can also use double quotes, e. g. if you need to expand parameters:



        VALUE="
        "type": "account",
        "customer_id": $ID,
        "customer_email": "$EMAIL",
        "


        Then you can use the variable value later on.



        echo -n "$VALUE" | python -m json.tool






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Apr 11 at 12:10









        David Foerster

        915616




        915616






















             

            draft saved


            draft discarded


























             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f436978%2fwhy-is-the-json-content-from-heredoc-not-parsable%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            How to check contact read email or not when send email to Individual?

            Bahrain

            Postfix configuration issue with fips on centos 7; mailgun relay