How does Bash handle quoting in the string replacement section of parameter expansion?
Clash Royale CLAN TAG#URR8PPP
up vote
4
down vote
favorite
Is there any consistent logic to it?
some-command "$somevariable//some pattern/'how does this get parsed?'"
I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.
bash quoting variable-substitution
add a comment |Â
up vote
4
down vote
favorite
Is there any consistent logic to it?
some-command "$somevariable//some pattern/'how does this get parsed?'"
I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.
bash quoting variable-substitution
Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
â Faheem Mitha
Dec 15 '17 at 8:11
add a comment |Â
up vote
4
down vote
favorite
up vote
4
down vote
favorite
Is there any consistent logic to it?
some-command "$somevariable//some pattern/'how does this get parsed?'"
I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.
bash quoting variable-substitution
Is there any consistent logic to it?
some-command "$somevariable//some pattern/'how does this get parsed?'"
I've posted some conclusions and raw tests below as an "answer" but they're not a full answer by any means. The Bash man page appears silent on the subject.
bash quoting variable-substitution
asked Dec 15 '17 at 5:58
Wildcard
22k855154
22k855154
Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
â Faheem Mitha
Dec 15 '17 at 8:11
add a comment |Â
Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
â Faheem Mitha
Dec 15 '17 at 8:11
Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
â Faheem Mitha
Dec 15 '17 at 8:11
Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
â Faheem Mitha
Dec 15 '17 at 8:11
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
4
down vote
accepted
As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha
(changelog):
zz. When using the pattern substitution word expansion, bash now runs the
replacement string through quote removal, since it allows quotes in that
string to act as escape characters. This is not backwards compatible, so
it can be disabled by setting the bash compatibility mode to 4.2.
And the description for shopt -s compat42
(online manual):
compat42
If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.
The quoting single-quotes example:
$ s=abc'def; echo "'$s//'/'\'''"
'abc'''def'
$ shopt -s compat42
$ s=abc'def; echo "'$s//'/'\'''"
'abc'\''def'
$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:
$ shopt -s compat42
$ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
'abc'''def'
$ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
'abc"'''"def'
The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/""
prints ac
. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x
outputs a""c
.
add a comment |Â
up vote
1
down vote
General rules by reverse engineering:
- Quotes must be coupled (completed)
- Quotes are preserved (included in the actual replacement)
- Backslashes are preserved if they come before an arbitrary letter
- Backslashes are preserved if they escape a single quote
- A backslash backslash sequence is reduced to one backslash even within single quotes
- You can't escape a single quote within single quotes
- Parameter expansion works inside single quotes the same as outside
- If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed
And a conclusion:
- There is absolutely no way to produce the literal sequence
'''
as a substitution through parameter expansion. - However, it is very easy to produce the literal sequence
"'''"
as a substitution.
Some raw tests follow.
[vagrant@localhost ~]$ echo "$0"
-bash
[vagrant@localhost ~]$ echo "$0//a/x"
-bxsh
[vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
-bsome long string with spacessh
[vagrant@localhost ~]$ echo "$0//a/"quoted string""
-b"quoted string"sh
[vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
> wat
> }"
-b"unfinished quote}"
wat
sh
[vagrant@localhost ~]$ echo "$0//a/"escaped quote"
-b"escaped quotesh
[vagrant@localhost ~]$ echo "$0//a/\escaped escape"
-bescaped escapesh
[vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
-b'escaped single quotesh
[vagrant@localhost ~]$ echo "$0//a/''"
-b''sh
[vagrant@localhost ~]$ echo "$0//a/''''"
-b''''sh
[vagrant@localhost ~]$ echo "$0//a/'''"
> a'b}c"d
-b'''}"
a'bshcd
[vagrant@localhost ~]$ echo "$0//a/'''"
> w'x}y"z
-b'''}"
w'xshyz
[vagrant@localhost ~]$ echo "$0//a/''\"a test''"
> ^C
[vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
-b'''"a test''sh
[vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
> w}x"y
-b"a test'$0'}"
wshxy
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
-b"a test'$0'sh
[vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
-b"a test'-bash'sh
[vagrant@localhost ~]$
I don't have time for a full answer right now, but here is a tip: Instead of testingecho "$0//bar/qux"
, testtemp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment totemp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignmentq=$...
(but not inlocal q=$...
orexport q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.
â mtraceur
Dec 15 '17 at 8:13
But also here is a minimal example of producing the literal'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes:q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes:q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents:q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
andq=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version producesabc'\''def
â Wildcard
Dec 15 '17 at 8:31
1
But here's one that works in both 4.4 and 3.2:qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)
â ilkkachu
Dec 15 '17 at 9:50
1
ksh
seems to work like the newer Bash, but myzsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
â ilkkachu
Dec 15 '17 at 10:15
 |Â
show 7 more comments
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha
(changelog):
zz. When using the pattern substitution word expansion, bash now runs the
replacement string through quote removal, since it allows quotes in that
string to act as escape characters. This is not backwards compatible, so
it can be disabled by setting the bash compatibility mode to 4.2.
And the description for shopt -s compat42
(online manual):
compat42
If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.
The quoting single-quotes example:
$ s=abc'def; echo "'$s//'/'\'''"
'abc'''def'
$ shopt -s compat42
$ s=abc'def; echo "'$s//'/'\'''"
'abc'\''def'
$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:
$ shopt -s compat42
$ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
'abc'''def'
$ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
'abc"'''"def'
The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/""
prints ac
. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x
outputs a""c
.
add a comment |Â
up vote
4
down vote
accepted
As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha
(changelog):
zz. When using the pattern substitution word expansion, bash now runs the
replacement string through quote removal, since it allows quotes in that
string to act as escape characters. This is not backwards compatible, so
it can be disabled by setting the bash compatibility mode to 4.2.
And the description for shopt -s compat42
(online manual):
compat42
If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.
The quoting single-quotes example:
$ s=abc'def; echo "'$s//'/'\'''"
'abc'''def'
$ shopt -s compat42
$ s=abc'def; echo "'$s//'/'\'''"
'abc'\''def'
$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:
$ shopt -s compat42
$ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
'abc'''def'
$ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
'abc"'''"def'
The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/""
prints ac
. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x
outputs a""c
.
add a comment |Â
up vote
4
down vote
accepted
up vote
4
down vote
accepted
As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha
(changelog):
zz. When using the pattern substitution word expansion, bash now runs the
replacement string through quote removal, since it allows quotes in that
string to act as escape characters. This is not backwards compatible, so
it can be disabled by setting the bash compatibility mode to 4.2.
And the description for shopt -s compat42
(online manual):
compat42
If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.
The quoting single-quotes example:
$ s=abc'def; echo "'$s//'/'\'''"
'abc'''def'
$ shopt -s compat42
$ s=abc'def; echo "'$s//'/'\'''"
'abc'\''def'
$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:
$ shopt -s compat42
$ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
'abc'''def'
$ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
'abc"'''"def'
The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/""
prints ac
. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x
outputs a""c
.
As discussed in the comments, this seems to have changed between versions of Bash. I think this is the relevant change in bash-4.3-alpha
(changelog):
zz. When using the pattern substitution word expansion, bash now runs the
replacement string through quote removal, since it allows quotes in that
string to act as escape characters. This is not backwards compatible, so
it can be disabled by setting the bash compatibility mode to 4.2.
And the description for shopt -s compat42
(online manual):
compat42
If set, bash does not process the replacement string in the pattern substitution word expansion using quote removal.
The quoting single-quotes example:
$ s=abc'def; echo "'$s//'/'\'''"
'abc'''def'
$ shopt -s compat42
$ s=abc'def; echo "'$s//'/'\'''"
'abc'\''def'
$ bash --version | head -1
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Workaround: put the replacement string in a variable, and don't use quotes inside the replacement:
$ shopt -s compat42
$ qq="'''"; s=abc'def; echo "'$s//'/$qq'";
'abc'''def'
$ qq="'''"; s=abc'def; echo "'$s//'/"$qq"'";
'abc"'''"def'
The funny thing is, that if the expansion is unquoted, then the quotes are removed after the substitution, in all versions. That is s=abc; echo $s/b/""
prints ac
. This of course doesn't happen with other expansions, e.g. s='a""c' ; echo $s%x
outputs a""c
.
edited Dec 15 '17 at 16:10
answered Dec 15 '17 at 12:00
ilkkachu
49.9k674137
49.9k674137
add a comment |Â
add a comment |Â
up vote
1
down vote
General rules by reverse engineering:
- Quotes must be coupled (completed)
- Quotes are preserved (included in the actual replacement)
- Backslashes are preserved if they come before an arbitrary letter
- Backslashes are preserved if they escape a single quote
- A backslash backslash sequence is reduced to one backslash even within single quotes
- You can't escape a single quote within single quotes
- Parameter expansion works inside single quotes the same as outside
- If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed
And a conclusion:
- There is absolutely no way to produce the literal sequence
'''
as a substitution through parameter expansion. - However, it is very easy to produce the literal sequence
"'''"
as a substitution.
Some raw tests follow.
[vagrant@localhost ~]$ echo "$0"
-bash
[vagrant@localhost ~]$ echo "$0//a/x"
-bxsh
[vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
-bsome long string with spacessh
[vagrant@localhost ~]$ echo "$0//a/"quoted string""
-b"quoted string"sh
[vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
> wat
> }"
-b"unfinished quote}"
wat
sh
[vagrant@localhost ~]$ echo "$0//a/"escaped quote"
-b"escaped quotesh
[vagrant@localhost ~]$ echo "$0//a/\escaped escape"
-bescaped escapesh
[vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
-b'escaped single quotesh
[vagrant@localhost ~]$ echo "$0//a/''"
-b''sh
[vagrant@localhost ~]$ echo "$0//a/''''"
-b''''sh
[vagrant@localhost ~]$ echo "$0//a/'''"
> a'b}c"d
-b'''}"
a'bshcd
[vagrant@localhost ~]$ echo "$0//a/'''"
> w'x}y"z
-b'''}"
w'xshyz
[vagrant@localhost ~]$ echo "$0//a/''\"a test''"
> ^C
[vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
-b'''"a test''sh
[vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
> w}x"y
-b"a test'$0'}"
wshxy
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
-b"a test'$0'sh
[vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
-b"a test'-bash'sh
[vagrant@localhost ~]$
I don't have time for a full answer right now, but here is a tip: Instead of testingecho "$0//bar/qux"
, testtemp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment totemp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignmentq=$...
(but not inlocal q=$...
orexport q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.
â mtraceur
Dec 15 '17 at 8:13
But also here is a minimal example of producing the literal'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes:q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes:q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents:q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
andq=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version producesabc'\''def
â Wildcard
Dec 15 '17 at 8:31
1
But here's one that works in both 4.4 and 3.2:qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)
â ilkkachu
Dec 15 '17 at 9:50
1
ksh
seems to work like the newer Bash, but myzsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
â ilkkachu
Dec 15 '17 at 10:15
 |Â
show 7 more comments
up vote
1
down vote
General rules by reverse engineering:
- Quotes must be coupled (completed)
- Quotes are preserved (included in the actual replacement)
- Backslashes are preserved if they come before an arbitrary letter
- Backslashes are preserved if they escape a single quote
- A backslash backslash sequence is reduced to one backslash even within single quotes
- You can't escape a single quote within single quotes
- Parameter expansion works inside single quotes the same as outside
- If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed
And a conclusion:
- There is absolutely no way to produce the literal sequence
'''
as a substitution through parameter expansion. - However, it is very easy to produce the literal sequence
"'''"
as a substitution.
Some raw tests follow.
[vagrant@localhost ~]$ echo "$0"
-bash
[vagrant@localhost ~]$ echo "$0//a/x"
-bxsh
[vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
-bsome long string with spacessh
[vagrant@localhost ~]$ echo "$0//a/"quoted string""
-b"quoted string"sh
[vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
> wat
> }"
-b"unfinished quote}"
wat
sh
[vagrant@localhost ~]$ echo "$0//a/"escaped quote"
-b"escaped quotesh
[vagrant@localhost ~]$ echo "$0//a/\escaped escape"
-bescaped escapesh
[vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
-b'escaped single quotesh
[vagrant@localhost ~]$ echo "$0//a/''"
-b''sh
[vagrant@localhost ~]$ echo "$0//a/''''"
-b''''sh
[vagrant@localhost ~]$ echo "$0//a/'''"
> a'b}c"d
-b'''}"
a'bshcd
[vagrant@localhost ~]$ echo "$0//a/'''"
> w'x}y"z
-b'''}"
w'xshyz
[vagrant@localhost ~]$ echo "$0//a/''\"a test''"
> ^C
[vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
-b'''"a test''sh
[vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
> w}x"y
-b"a test'$0'}"
wshxy
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
-b"a test'$0'sh
[vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
-b"a test'-bash'sh
[vagrant@localhost ~]$
I don't have time for a full answer right now, but here is a tip: Instead of testingecho "$0//bar/qux"
, testtemp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment totemp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignmentq=$...
(but not inlocal q=$...
orexport q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.
â mtraceur
Dec 15 '17 at 8:13
But also here is a minimal example of producing the literal'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes:q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes:q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents:q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
andq=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version producesabc'\''def
â Wildcard
Dec 15 '17 at 8:31
1
But here's one that works in both 4.4 and 3.2:qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)
â ilkkachu
Dec 15 '17 at 9:50
1
ksh
seems to work like the newer Bash, but myzsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
â ilkkachu
Dec 15 '17 at 10:15
 |Â
show 7 more comments
up vote
1
down vote
up vote
1
down vote
General rules by reverse engineering:
- Quotes must be coupled (completed)
- Quotes are preserved (included in the actual replacement)
- Backslashes are preserved if they come before an arbitrary letter
- Backslashes are preserved if they escape a single quote
- A backslash backslash sequence is reduced to one backslash even within single quotes
- You can't escape a single quote within single quotes
- Parameter expansion works inside single quotes the same as outside
- If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed
And a conclusion:
- There is absolutely no way to produce the literal sequence
'''
as a substitution through parameter expansion. - However, it is very easy to produce the literal sequence
"'''"
as a substitution.
Some raw tests follow.
[vagrant@localhost ~]$ echo "$0"
-bash
[vagrant@localhost ~]$ echo "$0//a/x"
-bxsh
[vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
-bsome long string with spacessh
[vagrant@localhost ~]$ echo "$0//a/"quoted string""
-b"quoted string"sh
[vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
> wat
> }"
-b"unfinished quote}"
wat
sh
[vagrant@localhost ~]$ echo "$0//a/"escaped quote"
-b"escaped quotesh
[vagrant@localhost ~]$ echo "$0//a/\escaped escape"
-bescaped escapesh
[vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
-b'escaped single quotesh
[vagrant@localhost ~]$ echo "$0//a/''"
-b''sh
[vagrant@localhost ~]$ echo "$0//a/''''"
-b''''sh
[vagrant@localhost ~]$ echo "$0//a/'''"
> a'b}c"d
-b'''}"
a'bshcd
[vagrant@localhost ~]$ echo "$0//a/'''"
> w'x}y"z
-b'''}"
w'xshyz
[vagrant@localhost ~]$ echo "$0//a/''\"a test''"
> ^C
[vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
-b'''"a test''sh
[vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
> w}x"y
-b"a test'$0'}"
wshxy
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
-b"a test'$0'sh
[vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
-b"a test'-bash'sh
[vagrant@localhost ~]$
General rules by reverse engineering:
- Quotes must be coupled (completed)
- Quotes are preserved (included in the actual replacement)
- Backslashes are preserved if they come before an arbitrary letter
- Backslashes are preserved if they escape a single quote
- A backslash backslash sequence is reduced to one backslash even within single quotes
- You can't escape a single quote within single quotes
- Parameter expansion works inside single quotes the same as outside
- If a dollar sign is escaped with a backslash the dollar sign is preserved literally and the backslash is removed
And a conclusion:
- There is absolutely no way to produce the literal sequence
'''
as a substitution through parameter expansion. - However, it is very easy to produce the literal sequence
"'''"
as a substitution.
Some raw tests follow.
[vagrant@localhost ~]$ echo "$0"
-bash
[vagrant@localhost ~]$ echo "$0//a/x"
-bxsh
[vagrant@localhost ~]$ echo "$0//a/some long string with spaces"
-bsome long string with spacessh
[vagrant@localhost ~]$ echo "$0//a/"quoted string""
-b"quoted string"sh
[vagrant@localhost ~]$ echo "$0//a/"unfinished quote"
> wat
> }"
-b"unfinished quote}"
wat
sh
[vagrant@localhost ~]$ echo "$0//a/"escaped quote"
-b"escaped quotesh
[vagrant@localhost ~]$ echo "$0//a/\escaped escape"
-bescaped escapesh
[vagrant@localhost ~]$ echo "$0//a/'escaped single quote"
-b'escaped single quotesh
[vagrant@localhost ~]$ echo "$0//a/''"
-b''sh
[vagrant@localhost ~]$ echo "$0//a/''''"
-b''''sh
[vagrant@localhost ~]$ echo "$0//a/'''"
> a'b}c"d
-b'''}"
a'bshcd
[vagrant@localhost ~]$ echo "$0//a/'''"
> w'x}y"z
-b'''}"
w'xshyz
[vagrant@localhost ~]$ echo "$0//a/''\"a test''"
> ^C
[vagrant@localhost ~]$ echo "$0//a/'''\"a test''"
-b'''"a test''sh
[vagrant@localhost ~]$ echo "$0//a/'''\"a test'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
> w}x"y
-b"a test'$0'}"
wshxy
[vagrant@localhost ~]$ echo "$0//a/\"a test'$0'"
-b"a test'$0'sh
[vagrant@localhost ~]$ echo "$0//a/\"a test'\'$0'"
> ^C
[vagrant@localhost ~]$ echo "$0//a/\"a test'\$0'"
-b"a test'-bash'sh
[vagrant@localhost ~]$
answered Dec 15 '17 at 5:58
Wildcard
22k855154
22k855154
I don't have time for a full answer right now, but here is a tip: Instead of testingecho "$0//bar/qux"
, testtemp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment totemp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignmentq=$...
(but not inlocal q=$...
orexport q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.
â mtraceur
Dec 15 '17 at 8:13
But also here is a minimal example of producing the literal'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes:q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes:q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents:q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
andq=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version producesabc'\''def
â Wildcard
Dec 15 '17 at 8:31
1
But here's one that works in both 4.4 and 3.2:qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)
â ilkkachu
Dec 15 '17 at 9:50
1
ksh
seems to work like the newer Bash, but myzsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
â ilkkachu
Dec 15 '17 at 10:15
 |Â
show 7 more comments
I don't have time for a full answer right now, but here is a tip: Instead of testingecho "$0//bar/qux"
, testtemp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment totemp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignmentq=$...
(but not inlocal q=$...
orexport q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.
â mtraceur
Dec 15 '17 at 8:13
But also here is a minimal example of producing the literal'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes:q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes:q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents:q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
andq=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version producesabc'\''def
â Wildcard
Dec 15 '17 at 8:31
1
But here's one that works in both 4.4 and 3.2:qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)
â ilkkachu
Dec 15 '17 at 9:50
1
ksh
seems to work like the newer Bash, but myzsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.
â ilkkachu
Dec 15 '17 at 10:15
I don't have time for a full answer right now, but here is a tip: Instead of testing
echo "$0//bar/qux"
, test temp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment to temp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$...
(but not in local q=$...
or export q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.â mtraceur
Dec 15 '17 at 8:13
I don't have time for a full answer right now, but here is a tip: Instead of testing
echo "$0//bar/qux"
, test temp=$0//bar/qux; echo "$temp"
(notice lack of double quotes in assignment to temp
). You're fighting a very unintuitive nesting combination of two layers of quoting rules - the double quotes around an expansion as part of a command, and the quoting of strings inside the substring replacement: in a plain variable assignment q=$...
(but not in local q=$...
or export q=$...
) expansions happen as if quoted, so you can work with the substring replacement by itself.â mtraceur
Dec 15 '17 at 8:13
But also here is a minimal example of producing the literal
'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
But also here is a minimal example of producing the literal
'''
as part of a substitution inline/nested in a double-quoted variable expansion using gratuitous backslash escapes: q=abc'def; echo "$q"; echo "$q//'/'\''"
. And another minimal example using double-quotes-in-double-quotes: q=abc'def; echo "$q"; echo "$q//"'"/"'''""
. Like I said, the nesting rules are unintuitive, which is why I prefer the temp-variable equivalents: q=abc'def; echo "$q"; q=$q//'/'\''; echo "$q"
and q=abc'def; echo "$q"; q=$q//"'"/"'''"; echo "$q"
â mtraceur
Dec 15 '17 at 8:23
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces
abc'\''def
â Wildcard
Dec 15 '17 at 8:31
@mtraceur, what version of what shell are you using? With bash 4.1.2 your first version produces
abc'\''def
â Wildcard
Dec 15 '17 at 8:31
1
1
But here's one that works in both 4.4 and 3.2:
qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)â ilkkachu
Dec 15 '17 at 9:50
But here's one that works in both 4.4 and 3.2:
qq="'''"; s=abc'def; echo "$s"; echo "'$s//'/$qq'";
(i.e. put the replacement string in variable)â ilkkachu
Dec 15 '17 at 9:50
1
1
ksh
seems to work like the newer Bash, but my zsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.â ilkkachu
Dec 15 '17 at 10:15
ksh
seems to work like the newer Bash, but my zsh
doesn't seem to take the quotes any more specially than other characters (it probably has a flag for that anyway)... rats, you nerd-sniped me.â ilkkachu
Dec 15 '17 at 10:15
 |Â
show 7 more comments
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f410983%2fhow-does-bash-handle-quoting-in-the-string-replacement-section-of-parameter-expa%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Some explanation of context might be helpful to readers. I see this question may originate from unix.stackexchange.com/q/410961/4671
â Faheem Mitha
Dec 15 '17 at 8:11