Is there a difference between cmd_drain < <(cmd_src) and cmd_drain <<< “$(cmd_src)”?
Clash Royale CLAN TAG#URR8PPP
The cmd | read -r var1 var2
construct famously does not work in bash because the read command is executed in a subshell due to piping. I used to use read -r var1 var2 <<< "$(cmd)"
to get around this, but recently I learned about the cmd_drain < <(cmd_src)
construct, which seems to work just as well: read -r var1 var2 < <$(cmd)
.
Is there a difference between these two solutions? There does not seem to be any difference in the trivial case:
$ hd < <(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
$ hd <<< $(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
I also tried some special characters and got the same results. My gut feeling is that the result will always be the same expect that cmd_drain <<< "$(cmd_src)"
will first run cmd_src
and buffer the whole result in memory before feeding it to cmd_drain
, while cmd_drain < <(cmd_src)
will continously feed the output of cmd_src
into cmd_drain
. I assume it behaves like cmd_src | cmd_drain
except that cmd_src
will be run in a sub-shell instead of cmd_drain
. Is my assumption correct?
Bonus question: Is quoting necessary around the $()
construct?
bash
add a comment |
The cmd | read -r var1 var2
construct famously does not work in bash because the read command is executed in a subshell due to piping. I used to use read -r var1 var2 <<< "$(cmd)"
to get around this, but recently I learned about the cmd_drain < <(cmd_src)
construct, which seems to work just as well: read -r var1 var2 < <$(cmd)
.
Is there a difference between these two solutions? There does not seem to be any difference in the trivial case:
$ hd < <(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
$ hd <<< $(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
I also tried some special characters and got the same results. My gut feeling is that the result will always be the same expect that cmd_drain <<< "$(cmd_src)"
will first run cmd_src
and buffer the whole result in memory before feeding it to cmd_drain
, while cmd_drain < <(cmd_src)
will continously feed the output of cmd_src
into cmd_drain
. I assume it behaves like cmd_src | cmd_drain
except that cmd_src
will be run in a sub-shell instead of cmd_drain
. Is my assumption correct?
Bonus question: Is quoting necessary around the $()
construct?
bash
add a comment |
The cmd | read -r var1 var2
construct famously does not work in bash because the read command is executed in a subshell due to piping. I used to use read -r var1 var2 <<< "$(cmd)"
to get around this, but recently I learned about the cmd_drain < <(cmd_src)
construct, which seems to work just as well: read -r var1 var2 < <$(cmd)
.
Is there a difference between these two solutions? There does not seem to be any difference in the trivial case:
$ hd < <(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
$ hd <<< $(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
I also tried some special characters and got the same results. My gut feeling is that the result will always be the same expect that cmd_drain <<< "$(cmd_src)"
will first run cmd_src
and buffer the whole result in memory before feeding it to cmd_drain
, while cmd_drain < <(cmd_src)
will continously feed the output of cmd_src
into cmd_drain
. I assume it behaves like cmd_src | cmd_drain
except that cmd_src
will be run in a sub-shell instead of cmd_drain
. Is my assumption correct?
Bonus question: Is quoting necessary around the $()
construct?
bash
The cmd | read -r var1 var2
construct famously does not work in bash because the read command is executed in a subshell due to piping. I used to use read -r var1 var2 <<< "$(cmd)"
to get around this, but recently I learned about the cmd_drain < <(cmd_src)
construct, which seems to work just as well: read -r var1 var2 < <$(cmd)
.
Is there a difference between these two solutions? There does not seem to be any difference in the trivial case:
$ hd < <(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
$ hd <<< $(echo Hello)
00000000 48 65 6c 6c 6f 0a |Hello.|
00000006
I also tried some special characters and got the same results. My gut feeling is that the result will always be the same expect that cmd_drain <<< "$(cmd_src)"
will first run cmd_src
and buffer the whole result in memory before feeding it to cmd_drain
, while cmd_drain < <(cmd_src)
will continously feed the output of cmd_src
into cmd_drain
. I assume it behaves like cmd_src | cmd_drain
except that cmd_src
will be run in a sub-shell instead of cmd_drain
. Is my assumption correct?
Bonus question: Is quoting necessary around the $()
construct?
bash
bash
asked Jan 15 at 10:04
ZoltanZoltan
253112
253112
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Yes, your assumption is correct. In cmd_drain < <(cmd_src)
(aka Process Substitution, combined with normal redirection), Bash will replace <(cmd_src)
with the path to a file, from which the output of cmd_src
can be read. From the docs:
The process list is run asynchronously, and its input or output
appears as a filename. This filename is passed as an argument to the
current command as the result of the expansion. If the>(list)
form
is used, writing to the file will provide input for list. If the
<(list)
form is used, the file passed as an argument should be read
to obtain the output of list.
In cmd_drain <<< "$(cmd_src)"
, <<< ...
is treated like any other here-string, so:
The word undergoes tilde expansion, parameter and variable expansion,
command substitution, arithmetic expansion, and quote removal.
Pathname expansion and word splitting are not performed. The result is
supplied as a single string, with a newline appended, to the command
on its standard input [...]
So you don't need to quote $()
there, but specifically because the here string <<<
syntax doesn't do word splitting or filename expansion. Usually, you'd have to.
Note again the last sentence of the here string documentation - a newline is appended:
bash-5.0$ od -c <<< $(printf %s foo)
0000000 f o o n
0000004
bash-5.0$ od -c < <(printf %s foo)
0000000 f o o
0000003
Whether or not that matters is up to what you're running.
In hd <<< $(echo Hello)
, the command substitution removes the trailing newline output by echo
, and the here string adds a newline, effectively giving you the same output. But, as the above example shows, this removal/addition of newlines can be tricky, and you need not get exactly what cmd_src
output.
4
The implementation is substantively different in ways I don't see this answer currently addressing -- with<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with< <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.
– Charles Duffy
Jan 15 at 15:41
2
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
1
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoidedfork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).
– Charles Duffy
Jan 16 at 1:16
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f494574%2fis-there-a-difference-between-cmd-drain-cmd-src-and-cmd-drain-cmd-sr%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Yes, your assumption is correct. In cmd_drain < <(cmd_src)
(aka Process Substitution, combined with normal redirection), Bash will replace <(cmd_src)
with the path to a file, from which the output of cmd_src
can be read. From the docs:
The process list is run asynchronously, and its input or output
appears as a filename. This filename is passed as an argument to the
current command as the result of the expansion. If the>(list)
form
is used, writing to the file will provide input for list. If the
<(list)
form is used, the file passed as an argument should be read
to obtain the output of list.
In cmd_drain <<< "$(cmd_src)"
, <<< ...
is treated like any other here-string, so:
The word undergoes tilde expansion, parameter and variable expansion,
command substitution, arithmetic expansion, and quote removal.
Pathname expansion and word splitting are not performed. The result is
supplied as a single string, with a newline appended, to the command
on its standard input [...]
So you don't need to quote $()
there, but specifically because the here string <<<
syntax doesn't do word splitting or filename expansion. Usually, you'd have to.
Note again the last sentence of the here string documentation - a newline is appended:
bash-5.0$ od -c <<< $(printf %s foo)
0000000 f o o n
0000004
bash-5.0$ od -c < <(printf %s foo)
0000000 f o o
0000003
Whether or not that matters is up to what you're running.
In hd <<< $(echo Hello)
, the command substitution removes the trailing newline output by echo
, and the here string adds a newline, effectively giving you the same output. But, as the above example shows, this removal/addition of newlines can be tricky, and you need not get exactly what cmd_src
output.
4
The implementation is substantively different in ways I don't see this answer currently addressing -- with<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with< <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.
– Charles Duffy
Jan 15 at 15:41
2
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
1
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoidedfork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).
– Charles Duffy
Jan 16 at 1:16
add a comment |
Yes, your assumption is correct. In cmd_drain < <(cmd_src)
(aka Process Substitution, combined with normal redirection), Bash will replace <(cmd_src)
with the path to a file, from which the output of cmd_src
can be read. From the docs:
The process list is run asynchronously, and its input or output
appears as a filename. This filename is passed as an argument to the
current command as the result of the expansion. If the>(list)
form
is used, writing to the file will provide input for list. If the
<(list)
form is used, the file passed as an argument should be read
to obtain the output of list.
In cmd_drain <<< "$(cmd_src)"
, <<< ...
is treated like any other here-string, so:
The word undergoes tilde expansion, parameter and variable expansion,
command substitution, arithmetic expansion, and quote removal.
Pathname expansion and word splitting are not performed. The result is
supplied as a single string, with a newline appended, to the command
on its standard input [...]
So you don't need to quote $()
there, but specifically because the here string <<<
syntax doesn't do word splitting or filename expansion. Usually, you'd have to.
Note again the last sentence of the here string documentation - a newline is appended:
bash-5.0$ od -c <<< $(printf %s foo)
0000000 f o o n
0000004
bash-5.0$ od -c < <(printf %s foo)
0000000 f o o
0000003
Whether or not that matters is up to what you're running.
In hd <<< $(echo Hello)
, the command substitution removes the trailing newline output by echo
, and the here string adds a newline, effectively giving you the same output. But, as the above example shows, this removal/addition of newlines can be tricky, and you need not get exactly what cmd_src
output.
4
The implementation is substantively different in ways I don't see this answer currently addressing -- with<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with< <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.
– Charles Duffy
Jan 15 at 15:41
2
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
1
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoidedfork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).
– Charles Duffy
Jan 16 at 1:16
add a comment |
Yes, your assumption is correct. In cmd_drain < <(cmd_src)
(aka Process Substitution, combined with normal redirection), Bash will replace <(cmd_src)
with the path to a file, from which the output of cmd_src
can be read. From the docs:
The process list is run asynchronously, and its input or output
appears as a filename. This filename is passed as an argument to the
current command as the result of the expansion. If the>(list)
form
is used, writing to the file will provide input for list. If the
<(list)
form is used, the file passed as an argument should be read
to obtain the output of list.
In cmd_drain <<< "$(cmd_src)"
, <<< ...
is treated like any other here-string, so:
The word undergoes tilde expansion, parameter and variable expansion,
command substitution, arithmetic expansion, and quote removal.
Pathname expansion and word splitting are not performed. The result is
supplied as a single string, with a newline appended, to the command
on its standard input [...]
So you don't need to quote $()
there, but specifically because the here string <<<
syntax doesn't do word splitting or filename expansion. Usually, you'd have to.
Note again the last sentence of the here string documentation - a newline is appended:
bash-5.0$ od -c <<< $(printf %s foo)
0000000 f o o n
0000004
bash-5.0$ od -c < <(printf %s foo)
0000000 f o o
0000003
Whether or not that matters is up to what you're running.
In hd <<< $(echo Hello)
, the command substitution removes the trailing newline output by echo
, and the here string adds a newline, effectively giving you the same output. But, as the above example shows, this removal/addition of newlines can be tricky, and you need not get exactly what cmd_src
output.
Yes, your assumption is correct. In cmd_drain < <(cmd_src)
(aka Process Substitution, combined with normal redirection), Bash will replace <(cmd_src)
with the path to a file, from which the output of cmd_src
can be read. From the docs:
The process list is run asynchronously, and its input or output
appears as a filename. This filename is passed as an argument to the
current command as the result of the expansion. If the>(list)
form
is used, writing to the file will provide input for list. If the
<(list)
form is used, the file passed as an argument should be read
to obtain the output of list.
In cmd_drain <<< "$(cmd_src)"
, <<< ...
is treated like any other here-string, so:
The word undergoes tilde expansion, parameter and variable expansion,
command substitution, arithmetic expansion, and quote removal.
Pathname expansion and word splitting are not performed. The result is
supplied as a single string, with a newline appended, to the command
on its standard input [...]
So you don't need to quote $()
there, but specifically because the here string <<<
syntax doesn't do word splitting or filename expansion. Usually, you'd have to.
Note again the last sentence of the here string documentation - a newline is appended:
bash-5.0$ od -c <<< $(printf %s foo)
0000000 f o o n
0000004
bash-5.0$ od -c < <(printf %s foo)
0000000 f o o
0000003
Whether or not that matters is up to what you're running.
In hd <<< $(echo Hello)
, the command substitution removes the trailing newline output by echo
, and the here string adds a newline, effectively giving you the same output. But, as the above example shows, this removal/addition of newlines can be tricky, and you need not get exactly what cmd_src
output.
edited Jan 15 at 10:28
answered Jan 15 at 10:21
OlorinOlorin
3,2741417
3,2741417
4
The implementation is substantively different in ways I don't see this answer currently addressing -- with<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with< <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.
– Charles Duffy
Jan 15 at 15:41
2
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
1
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoidedfork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).
– Charles Duffy
Jan 16 at 1:16
add a comment |
4
The implementation is substantively different in ways I don't see this answer currently addressing -- with<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with< <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.
– Charles Duffy
Jan 15 at 15:41
2
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
1
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoidedfork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).
– Charles Duffy
Jan 16 at 1:16
4
4
The implementation is substantively different in ways I don't see this answer currently addressing -- with
<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with < <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.– Charles Duffy
Jan 15 at 15:41
The implementation is substantively different in ways I don't see this answer currently addressing -- with
<<<"$(...)"
, the content is collected as a whole written to a seekable temporary file (only available for use after completely finished generation); with < <(...)
, it's streamed over a FIFO (so content becomes available as it's generated, rather than needing the writing process to finish before the reading process can start) and never touches disk.– Charles Duffy
Jan 15 at 15:41
2
2
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy this difference was already mentioned in my question as an assumption - although the here-string creating a temporary file was not. In fact, I mistakenly assumed that the here-string will be buffered in memory, not on disk.
– Zoltan
Jan 15 at 17:09
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
@CharlesDuffy Thanks for the info, but is that documented behaviour? The undocumented side-effects of the particular implementation may change without notice, and for all I know, bash may use different implementations for different OS as well.
– Olorin
Jan 16 at 1:10
1
1
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoided
fork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).– Charles Duffy
Jan 16 at 1:16
@Olorin, undocumented, implementation-defined, and subject-to-change, indeed. (That said, on systems using tmpfs the temporary-file approach typically microbenchmarks faster than a process substitution generating the same output due to the avoided
fork()
, and there aren't any portable means I'm aware of to create a seekable file that isn't represented in the filesystem layer, so I'd be surprised to see much change any time soon; the approaches each have unique advantages provided by their present implementations).– Charles Duffy
Jan 16 at 1:16
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f494574%2fis-there-a-difference-between-cmd-drain-cmd-src-and-cmd-drain-cmd-sr%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown