Correct location for piping and redirecting output in find -exec?

Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).
How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the
findor theexeccommands on different occasions?On the same theme, what is the syntax if I want to use
teeto view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?
(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)
Update:
In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :
- outer
find: find all subdirs matching DIR_MATCH_TEXT inDIR1 - inner
find: for each matching subdir found in the outer loop (call it DIR2), execute the commandfind DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt
The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.
By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.
From replies so far I gather the outer find would have to use a shell call as the argument to -exec?
shell find freebsd io-redirection
add a comment |Â
up vote
0
down vote
favorite
I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).
How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the
findor theexeccommands on different occasions?On the same theme, what is the syntax if I want to use
teeto view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?
(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)
Update:
In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :
- outer
find: find all subdirs matching DIR_MATCH_TEXT inDIR1 - inner
find: for each matching subdir found in the outer loop (call it DIR2), execute the commandfind DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt
The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.
By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.
From replies so far I gather the outer find would have to use a shell call as the argument to -exec?
shell find freebsd io-redirection
1
Does this help? unix.stackexchange.com/questions/389705/â¦
â Kusalananda
Oct 3 '17 at 15:17
Duplicate question: unix.stackexchange.com/questions/111520/⦠. You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding bothstdoutANDstderroutput to the specified location for redirection.
â ILMostro_7
Oct 3 '17 at 15:18
These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
â Stilez
Oct 3 '17 at 15:25
1
@Stilez In the second case, you may move the redirection to the end of the command. It is not part of-execnor of the options tofind. See unix.stackexchange.com/questions/356118/â¦
â Kusalananda
Oct 3 '17 at 15:49
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).
How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the
findor theexeccommands on different occasions?On the same theme, what is the syntax if I want to use
teeto view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?
(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)
Update:
In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :
- outer
find: find all subdirs matching DIR_MATCH_TEXT inDIR1 - inner
find: for each matching subdir found in the outer loop (call it DIR2), execute the commandfind DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt
The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.
By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.
From replies so far I gather the outer find would have to use a shell call as the argument to -exec?
shell find freebsd io-redirection
I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).
How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the
findor theexeccommands on different occasions?On the same theme, what is the syntax if I want to use
teeto view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?
(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)
Update:
In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :
- outer
find: find all subdirs matching DIR_MATCH_TEXT inDIR1 - inner
find: for each matching subdir found in the outer loop (call it DIR2), execute the commandfind DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt
The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.
By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.
From replies so far I gather the outer find would have to use a shell call as the argument to -exec?
shell find freebsd io-redirection
shell find freebsd io-redirection
edited Oct 6 '17 at 7:56
asked Oct 3 '17 at 15:13
Stilez
441211
441211
1
Does this help? unix.stackexchange.com/questions/389705/â¦
â Kusalananda
Oct 3 '17 at 15:17
Duplicate question: unix.stackexchange.com/questions/111520/⦠. You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding bothstdoutANDstderroutput to the specified location for redirection.
â ILMostro_7
Oct 3 '17 at 15:18
These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
â Stilez
Oct 3 '17 at 15:25
1
@Stilez In the second case, you may move the redirection to the end of the command. It is not part of-execnor of the options tofind. See unix.stackexchange.com/questions/356118/â¦
â Kusalananda
Oct 3 '17 at 15:49
add a comment |Â
1
Does this help? unix.stackexchange.com/questions/389705/â¦
â Kusalananda
Oct 3 '17 at 15:17
Duplicate question: unix.stackexchange.com/questions/111520/⦠. You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding bothstdoutANDstderroutput to the specified location for redirection.
â ILMostro_7
Oct 3 '17 at 15:18
These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
â Stilez
Oct 3 '17 at 15:25
1
@Stilez In the second case, you may move the redirection to the end of the command. It is not part of-execnor of the options tofind. See unix.stackexchange.com/questions/356118/â¦
â Kusalananda
Oct 3 '17 at 15:49
1
1
Does this help? unix.stackexchange.com/questions/389705/â¦
â Kusalananda
Oct 3 '17 at 15:17
Does this help? unix.stackexchange.com/questions/389705/â¦
â Kusalananda
Oct 3 '17 at 15:17
Duplicate question: unix.stackexchange.com/questions/111520/⦠. You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both
stdout AND stderr output to the specified location for redirection.â ILMostro_7
Oct 3 '17 at 15:18
Duplicate question: unix.stackexchange.com/questions/111520/⦠. You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both
stdout AND stderr output to the specified location for redirection.â ILMostro_7
Oct 3 '17 at 15:18
These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
â Stilez
Oct 3 '17 at 15:25
These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
â Stilez
Oct 3 '17 at 15:25
1
1
@Stilez In the second case, you may move the redirection to the end of the command. It is not part of
-exec nor of the options to find. See unix.stackexchange.com/questions/356118/â¦â Kusalananda
Oct 3 '17 at 15:49
@Stilez In the second case, you may move the redirection to the end of the command. It is not part of
-exec nor of the options to find. See unix.stackexchange.com/questions/356118/â¦â Kusalananda
Oct 3 '17 at 15:49
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.
You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:
find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;
That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.
Pipes will work as well, with the same caveat of "one exec per file".
find . -type f -exec bash -c "ls -l $0 | grep something" ;
All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.
You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:
find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;
That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.
Pipes will work as well, with the same caveat of "one exec per file".
find . -type f -exec bash -c "ls -l $0 | grep something" ;
All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
add a comment |Â
up vote
2
down vote
accepted
I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.
You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:
find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;
That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.
Pipes will work as well, with the same caveat of "one exec per file".
find . -type f -exec bash -c "ls -l $0 | grep something" ;
All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
add a comment |Â
up vote
2
down vote
accepted
up vote
2
down vote
accepted
I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.
You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:
find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;
That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.
Pipes will work as well, with the same caveat of "one exec per file".
find . -type f -exec bash -c "ls -l $0 | grep something" ;
All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.
I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.
You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:
find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;
That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.
Pipes will work as well, with the same caveat of "one exec per file".
find . -type f -exec bash -c "ls -l $0 | grep something" ;
All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.
answered Oct 6 '17 at 4:50
Andy Dalton
4,7991520
4,7991520
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
add a comment |Â
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
â Stilez
Oct 6 '17 at 7:32
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
Updated question to explain the use-case. Is it possible?
â Stilez
Oct 6 '17 at 7:57
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f395872%2fcorrect-location-for-piping-and-redirecting-output-in-find-exec%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Does this help? unix.stackexchange.com/questions/389705/â¦
â Kusalananda
Oct 3 '17 at 15:17
Duplicate question: unix.stackexchange.com/questions/111520/⦠. You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both
stdoutANDstderroutput to the specified location for redirection.â ILMostro_7
Oct 3 '17 at 15:18
These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
â Stilez
Oct 3 '17 at 15:25
1
@Stilez In the second case, you may move the redirection to the end of the command. It is not part of
-execnor of the options tofind. See unix.stackexchange.com/questions/356118/â¦â Kusalananda
Oct 3 '17 at 15:49