How does Bash handle quoting in the string replacement section of parameter expansion?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
4
down vote

favorite
1












Is there any consistent logic to it?



some-command "$somevariable//some pattern/'how does this get parsed?'"


I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.







share|improve this question




















  • Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
    – Faheem Mitha
    Dec 15 '17 at 8:11














up vote
4
down vote

favorite
1












Is there any consistent logic to it?



some-command "$somevariable//some pattern/'how does this get parsed?'"


I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.







share|improve this question




















  • Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
    – Faheem Mitha
    Dec 15 '17 at 8:11












up vote
4
down vote

favorite
1









up vote
4
down vote

favorite
1






1





Is there any consistent logic to it?



some-command "$somevariable//some pattern/'how does this get parsed?'"


I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.







share|improve this question












Is there any consistent logic to it?



some-command "$somevariable//some pattern/'how does this get parsed?'"


I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.









share|improve this question











share|improve this question




share|improve this question










asked Dec 15 '17 at 5:58









Wildcard

22k855154




22k855154











  • Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
    – Faheem Mitha
    Dec 15 '17 at 8:11
















  • Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
    – Faheem Mitha
    Dec 15 '17 at 8:11















Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
– Faheem Mitha
Dec 15 '17 at 8:11




Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
– Faheem Mitha
Dec 15 '17 at 8:11










2 Answers
2






active

oldest

votes

















up vote
4
down vote



accepted










As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha (changelog):




zz. When using the pattern substitution word expansion, bash now runs the
replacement string through quote removal, since it allows quotes in that
string to act as escape characters. This is not backwards compatible, so
it can be disabled by setting the bash compatibility mode to 4.2.




And the description for shopt -s compat42 (online manual):




compat42

If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.




The quoting single-quotes example:



$ s=abc'def; echo "'$s//'/'\'''"
'abc'''def'

$ shopt -s compat42
$ s=abc'def; echo "'$s//'/'\'''"
'abc'\''def'

$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)


Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:



$ shopt -s compat42
$ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
'abc'''def'
$ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
'abc"'''"def'


The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/"" prints ac. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x outputs a""c.






share|improve this answer





























    up vote
    1
    down vote













    General rules by reverse engineering:



    • Quotes must be coupled (completed)

    • Quotes are preserved (included in the actual replacement)

    • Backslashes are preserved if they come before an arbitrary letter

    • Backslashes are preserved if they escape a single quote

    • A backslash backslash sequence is reduced to one backslash even within single quotes

    • You can't escape a single quote within single quotes

    • Parameter expansion works inside single quotes the same as outside

    • If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed

    And a conclusion:



    • There is absolutely no way to produce the literal sequence ''' as a substitution through parameter expansion.

    • However, it is very easy to produce the literal sequence "'''" as a substitution.

    Some raw tests follow.



    [vagrant@localhost ~]$ echo "$0"
    -bash
    [vagrant@localhost ~]$ echo "$0//a/x"
    -bxsh
    [vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
    -bsome long string with spacessh
    [vagrant@localhost ~]$ echo "$0//a/"quoted string""
    -b"quoted string"sh
    [vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
    > wat
    > }"
    -b"unfinished quote}"
    wat
    sh
    [vagrant@localhost ~]$ echo "$0//a/"escaped quote"
    -b"escaped quotesh
    [vagrant@localhost ~]$ echo "$0//a/\escaped escape"
    -bescaped escapesh
    [vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
    -b'escaped single quotesh
    [vagrant@localhost ~]$ echo "$0//a/''"
    -b''sh
    [vagrant@localhost ~]$ echo "$0//a/''''"
    -b''''sh
    [vagrant@localhost ~]$ echo "$0//a/'''"
    > a'b}c"d
    -b'''}"
    a'bshcd
    [vagrant@localhost ~]$ echo "$0//a/'''"
    > w'x}y"z
    -b'''}"
    w'xshyz
    [vagrant@localhost ~]$ echo "$0//a/''\"a test''"
    > ^C
    [vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
    -b'''"a test''sh
    [vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
    > ^C
    [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
    > w}x"y
    -b"a test'$0'}"
    wshxy
    [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
    -b"a test'$0'sh
    [vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
    > ^C
    [vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
    -b"a test'-bash'sh
    [vagrant@localhost ~]$





    share|improve this answer




















    • I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
      – mtraceur
      Dec 15 '17 at 8:13











    • But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
      – mtraceur
      Dec 15 '17 at 8:23










    • @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
      – Wildcard
      Dec 15 '17 at 8:31






    • 1




      But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
      – ilkkachu
      Dec 15 '17 at 9:50






    • 1




      ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
      – ilkkachu
      Dec 15 '17 at 10:15











    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );








     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f410983%2fhow-does-bash-handle-quoting-in-the-string-replacement-section-of-parameter-expa%23new-answer', 'question_page');

    );

    Post as a guest






























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    4
    down vote



    accepted










    As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha (changelog):




    zz. When using the pattern substitution word expansion, bash now runs the
    replacement string through quote removal, since it allows quotes in that
    string to act as escape characters. This is not backwards compatible, so
    it can be disabled by setting the bash compatibility mode to 4.2.




    And the description for shopt -s compat42 (online manual):




    compat42

    If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.




    The quoting single-quotes example:



    $ s=abc'def; echo "'$s//'/'\'''"
    'abc'''def'

    $ shopt -s compat42
    $ s=abc'def; echo "'$s//'/'\'''"
    'abc'\''def'

    $ bash --version | head -1
    GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)


    Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:



    $ shopt -s compat42
    $ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
    'abc'''def'
    $ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
    'abc"'''"def'


    The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/"" prints ac. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x outputs a""c.






    share|improve this answer


























      up vote
      4
      down vote



      accepted










      As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha (changelog):




      zz. When using the pattern substitution word expansion, bash now runs the
      replacement string through quote removal, since it allows quotes in that
      string to act as escape characters. This is not backwards compatible, so
      it can be disabled by setting the bash compatibility mode to 4.2.




      And the description for shopt -s compat42 (online manual):




      compat42

      If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.




      The quoting single-quotes example:



      $ s=abc'def; echo "'$s//'/'\'''"
      'abc'''def'

      $ shopt -s compat42
      $ s=abc'def; echo "'$s//'/'\'''"
      'abc'\''def'

      $ bash --version | head -1
      GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)


      Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:



      $ shopt -s compat42
      $ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
      'abc'''def'
      $ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
      'abc"'''"def'


      The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/"" prints ac. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x outputs a""c.






      share|improve this answer
























        up vote
        4
        down vote



        accepted







        up vote
        4
        down vote



        accepted






        As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha (changelog):




        zz. When using the pattern substitution word expansion, bash now runs the
        replacement string through quote removal, since it allows quotes in that
        string to act as escape characters. This is not backwards compatible, so
        it can be disabled by setting the bash compatibility mode to 4.2.




        And the description for shopt -s compat42 (online manual):




        compat42

        If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.




        The quoting single-quotes example:



        $ s=abc'def; echo "'$s//'/'\'''"
        'abc'''def'

        $ shopt -s compat42
        $ s=abc'def; echo "'$s//'/'\'''"
        'abc'\''def'

        $ bash --version | head -1
        GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)


        Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:



        $ shopt -s compat42
        $ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
        'abc'''def'
        $ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
        'abc"'''"def'


        The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/"" prints ac. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x outputs a""c.






        share|improve this answer














        As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha (changelog):




        zz. When using the pattern substitution word expansion, bash now runs the
        replacement string through quote removal, since it allows quotes in that
        string to act as escape characters. This is not backwards compatible, so
        it can be disabled by setting the bash compatibility mode to 4.2.




        And the description for shopt -s compat42 (online manual):




        compat42

        If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.




        The quoting single-quotes example:



        $ s=abc'def; echo "'$s//'/'\'''"
        'abc'''def'

        $ shopt -s compat42
        $ s=abc'def; echo "'$s//'/'\'''"
        'abc'\''def'

        $ bash --version | head -1
        GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)


        Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:



        $ shopt -s compat42
        $ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
        'abc'''def'
        $ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
        'abc"'''"def'


        The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/"" prints ac. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x outputs a""c.







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Dec 15 '17 at 16:10

























        answered Dec 15 '17 at 12:00









        ilkkachu

        49.9k674137




        49.9k674137






















            up vote
            1
            down vote













            General rules by reverse engineering:



            • Quotes must be coupled (completed)

            • Quotes are preserved (included in the actual replacement)

            • Backslashes are preserved if they come before an arbitrary letter

            • Backslashes are preserved if they escape a single quote

            • A backslash backslash sequence is reduced to one backslash even within single quotes

            • You can't escape a single quote within single quotes

            • Parameter expansion works inside single quotes the same as outside

            • If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed

            And a conclusion:



            • There is absolutely no way to produce the literal sequence ''' as a substitution through parameter expansion.

            • However, it is very easy to produce the literal sequence "'''" as a substitution.

            Some raw tests follow.



            [vagrant@localhost ~]$ echo "$0"
            -bash
            [vagrant@localhost ~]$ echo "$0//a/x"
            -bxsh
            [vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
            -bsome long string with spacessh
            [vagrant@localhost ~]$ echo "$0//a/"quoted string""
            -b"quoted string"sh
            [vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
            > wat
            > }"
            -b"unfinished quote}"
            wat
            sh
            [vagrant@localhost ~]$ echo "$0//a/"escaped quote"
            -b"escaped quotesh
            [vagrant@localhost ~]$ echo "$0//a/\escaped escape"
            -bescaped escapesh
            [vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
            -b'escaped single quotesh
            [vagrant@localhost ~]$ echo "$0//a/''"
            -b''sh
            [vagrant@localhost ~]$ echo "$0//a/''''"
            -b''''sh
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > a'b}c"d
            -b'''}"
            a'bshcd
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > w'x}y"z
            -b'''}"
            w'xshyz
            [vagrant@localhost ~]$ echo "$0//a/''\"a test''"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
            -b'''"a test''sh
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            > w}x"y
            -b"a test'$0'}"
            wshxy
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            -b"a test'$0'sh
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
            -b"a test'-bash'sh
            [vagrant@localhost ~]$





            share|improve this answer




















            • I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
              – mtraceur
              Dec 15 '17 at 8:13











            • But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
              – mtraceur
              Dec 15 '17 at 8:23










            • @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
              – Wildcard
              Dec 15 '17 at 8:31






            • 1




              But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
              – ilkkachu
              Dec 15 '17 at 9:50






            • 1




              ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
              – ilkkachu
              Dec 15 '17 at 10:15















            up vote
            1
            down vote













            General rules by reverse engineering:



            • Quotes must be coupled (completed)

            • Quotes are preserved (included in the actual replacement)

            • Backslashes are preserved if they come before an arbitrary letter

            • Backslashes are preserved if they escape a single quote

            • A backslash backslash sequence is reduced to one backslash even within single quotes

            • You can't escape a single quote within single quotes

            • Parameter expansion works inside single quotes the same as outside

            • If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed

            And a conclusion:



            • There is absolutely no way to produce the literal sequence ''' as a substitution through parameter expansion.

            • However, it is very easy to produce the literal sequence "'''" as a substitution.

            Some raw tests follow.



            [vagrant@localhost ~]$ echo "$0"
            -bash
            [vagrant@localhost ~]$ echo "$0//a/x"
            -bxsh
            [vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
            -bsome long string with spacessh
            [vagrant@localhost ~]$ echo "$0//a/"quoted string""
            -b"quoted string"sh
            [vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
            > wat
            > }"
            -b"unfinished quote}"
            wat
            sh
            [vagrant@localhost ~]$ echo "$0//a/"escaped quote"
            -b"escaped quotesh
            [vagrant@localhost ~]$ echo "$0//a/\escaped escape"
            -bescaped escapesh
            [vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
            -b'escaped single quotesh
            [vagrant@localhost ~]$ echo "$0//a/''"
            -b''sh
            [vagrant@localhost ~]$ echo "$0//a/''''"
            -b''''sh
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > a'b}c"d
            -b'''}"
            a'bshcd
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > w'x}y"z
            -b'''}"
            w'xshyz
            [vagrant@localhost ~]$ echo "$0//a/''\"a test''"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
            -b'''"a test''sh
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            > w}x"y
            -b"a test'$0'}"
            wshxy
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            -b"a test'$0'sh
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
            -b"a test'-bash'sh
            [vagrant@localhost ~]$





            share|improve this answer




















            • I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
              – mtraceur
              Dec 15 '17 at 8:13











            • But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
              – mtraceur
              Dec 15 '17 at 8:23










            • @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
              – Wildcard
              Dec 15 '17 at 8:31






            • 1




              But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
              – ilkkachu
              Dec 15 '17 at 9:50






            • 1




              ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
              – ilkkachu
              Dec 15 '17 at 10:15













            up vote
            1
            down vote










            up vote
            1
            down vote









            General rules by reverse engineering:



            • Quotes must be coupled (completed)

            • Quotes are preserved (included in the actual replacement)

            • Backslashes are preserved if they come before an arbitrary letter

            • Backslashes are preserved if they escape a single quote

            • A backslash backslash sequence is reduced to one backslash even within single quotes

            • You can't escape a single quote within single quotes

            • Parameter expansion works inside single quotes the same as outside

            • If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed

            And a conclusion:



            • There is absolutely no way to produce the literal sequence ''' as a substitution through parameter expansion.

            • However, it is very easy to produce the literal sequence "'''" as a substitution.

            Some raw tests follow.



            [vagrant@localhost ~]$ echo "$0"
            -bash
            [vagrant@localhost ~]$ echo "$0//a/x"
            -bxsh
            [vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
            -bsome long string with spacessh
            [vagrant@localhost ~]$ echo "$0//a/"quoted string""
            -b"quoted string"sh
            [vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
            > wat
            > }"
            -b"unfinished quote}"
            wat
            sh
            [vagrant@localhost ~]$ echo "$0//a/"escaped quote"
            -b"escaped quotesh
            [vagrant@localhost ~]$ echo "$0//a/\escaped escape"
            -bescaped escapesh
            [vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
            -b'escaped single quotesh
            [vagrant@localhost ~]$ echo "$0//a/''"
            -b''sh
            [vagrant@localhost ~]$ echo "$0//a/''''"
            -b''''sh
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > a'b}c"d
            -b'''}"
            a'bshcd
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > w'x}y"z
            -b'''}"
            w'xshyz
            [vagrant@localhost ~]$ echo "$0//a/''\"a test''"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
            -b'''"a test''sh
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            > w}x"y
            -b"a test'$0'}"
            wshxy
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            -b"a test'$0'sh
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
            -b"a test'-bash'sh
            [vagrant@localhost ~]$





            share|improve this answer












            General rules by reverse engineering:



            • Quotes must be coupled (completed)

            • Quotes are preserved (included in the actual replacement)

            • Backslashes are preserved if they come before an arbitrary letter

            • Backslashes are preserved if they escape a single quote

            • A backslash backslash sequence is reduced to one backslash even within single quotes

            • You can't escape a single quote within single quotes

            • Parameter expansion works inside single quotes the same as outside

            • If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed

            And a conclusion:



            • There is absolutely no way to produce the literal sequence ''' as a substitution through parameter expansion.

            • However, it is very easy to produce the literal sequence "'''" as a substitution.

            Some raw tests follow.



            [vagrant@localhost ~]$ echo "$0"
            -bash
            [vagrant@localhost ~]$ echo "$0//a/x"
            -bxsh
            [vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
            -bsome long string with spacessh
            [vagrant@localhost ~]$ echo "$0//a/"quoted string""
            -b"quoted string"sh
            [vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
            > wat
            > }"
            -b"unfinished quote}"
            wat
            sh
            [vagrant@localhost ~]$ echo "$0//a/"escaped quote"
            -b"escaped quotesh
            [vagrant@localhost ~]$ echo "$0//a/\escaped escape"
            -bescaped escapesh
            [vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
            -b'escaped single quotesh
            [vagrant@localhost ~]$ echo "$0//a/''"
            -b''sh
            [vagrant@localhost ~]$ echo "$0//a/''''"
            -b''''sh
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > a'b}c"d
            -b'''}"
            a'bshcd
            [vagrant@localhost ~]$ echo "$0//a/'''"
            > w'x}y"z
            -b'''}"
            w'xshyz
            [vagrant@localhost ~]$ echo "$0//a/''\"a test''"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
            -b'''"a test''sh
            [vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            > w}x"y
            -b"a test'$0'}"
            wshxy
            [vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
            -b"a test'$0'sh
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
            > ^C
            [vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
            -b"a test'-bash'sh
            [vagrant@localhost ~]$






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Dec 15 '17 at 5:58









            Wildcard

            22k855154




            22k855154











            • I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
              – mtraceur
              Dec 15 '17 at 8:13











            • But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
              – mtraceur
              Dec 15 '17 at 8:23










            • @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
              – Wildcard
              Dec 15 '17 at 8:31






            • 1




              But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
              – ilkkachu
              Dec 15 '17 at 9:50






            • 1




              ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
              – ilkkachu
              Dec 15 '17 at 10:15

















            • I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
              – mtraceur
              Dec 15 '17 at 8:13











            • But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
              – mtraceur
              Dec 15 '17 at 8:23










            • @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
              – Wildcard
              Dec 15 '17 at 8:31






            • 1




              But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
              – ilkkachu
              Dec 15 '17 at 9:50






            • 1




              ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
              – ilkkachu
              Dec 15 '17 at 10:15
















            I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
            – mtraceur
            Dec 15 '17 at 8:13





            I don't have time for a full answer right now, but here is a tip: Instead of testing echo "$0//bar/qux", test temp=$0//bar/qux; echo "$temp" (notice lack of double quotes in assignment to temp). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$... (but not in local q=$... or export q=$...) expansions happen as if quoted, so you can work with the substring replacement by itself.
            – mtraceur
            Dec 15 '17 at 8:13













            But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
            – mtraceur
            Dec 15 '17 at 8:23




            But also here is a minimal example of producing the literal ''' as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''". And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''"". Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q" and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
            – mtraceur
            Dec 15 '17 at 8:23












            @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
            – Wildcard
            Dec 15 '17 at 8:31




            @mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces abc'\''def
            – Wildcard
            Dec 15 '17 at 8:31




            1




            1




            But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
            – ilkkachu
            Dec 15 '17 at 9:50




            But here's one that works in both 4.4 and 3.2: qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'"; (i.e. put the replacement string in variable)
            – ilkkachu
            Dec 15 '17 at 9:50




            1




            1




            ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
            – ilkkachu
            Dec 15 '17 at 10:15





            ksh seems to work like the newer Bash, but my zsh doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
            – ilkkachu
            Dec 15 '17 at 10:15













             

            draft saved


            draft discarded


























             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f410983%2fhow-does-bash-handle-quoting-in-the-string-replacement-section-of-parameter-expa%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            How to check contact read email or not when send email to Individual?

            Bahrain

            Postfix configuration issue with fips on centos 7; mailgun relay