Correct location for piping and redirecting output in find -exec?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite












I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).



  1. How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the find or the exec commands on different occasions?


  2. On the same theme, what is the syntax if I want to use tee to view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?


(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)



Update:



In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :



  • outer find: find all subdirs matching DIR_MATCH_TEXT in DIR1

  • inner find: for each matching subdir found in the outer loop (call it DIR2), execute the command
    find DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt

The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.



By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.



From replies so far I gather the outer find would have to use a shell call as the argument to -exec?










share|improve this question



















  • 1




    Does this help? unix.stackexchange.com/questions/389705/…
    – Kusalananda
    Oct 3 '17 at 15:17










  • Duplicate question: unix.stackexchange.com/questions/111520/… . You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both stdout AND stderr output to the specified location for redirection.
    – ILMostro_7
    Oct 3 '17 at 15:18











  • These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
    – Stilez
    Oct 3 '17 at 15:25







  • 1




    @Stilez In the second case, you may move the redirection to the end of the command. It is not part of -exec nor of the options to find. See unix.stackexchange.com/questions/356118/…
    – Kusalananda
    Oct 3 '17 at 15:49














up vote
0
down vote

favorite












I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).



  1. How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the find or the exec commands on different occasions?


  2. On the same theme, what is the syntax if I want to use tee to view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?


(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)



Update:



In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :



  • outer find: find all subdirs matching DIR_MATCH_TEXT in DIR1

  • inner find: for each matching subdir found in the outer loop (call it DIR2), execute the command
    find DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt

The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.



By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.



From replies so far I gather the outer find would have to use a shell call as the argument to -exec?










share|improve this question



















  • 1




    Does this help? unix.stackexchange.com/questions/389705/…
    – Kusalananda
    Oct 3 '17 at 15:17










  • Duplicate question: unix.stackexchange.com/questions/111520/… . You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both stdout AND stderr output to the specified location for redirection.
    – ILMostro_7
    Oct 3 '17 at 15:18











  • These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
    – Stilez
    Oct 3 '17 at 15:25







  • 1




    @Stilez In the second case, you may move the redirection to the end of the command. It is not part of -exec nor of the options to find. See unix.stackexchange.com/questions/356118/…
    – Kusalananda
    Oct 3 '17 at 15:49












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).



  1. How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the find or the exec commands on different occasions?


  2. On the same theme, what is the syntax if I want to use tee to view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?


(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)



Update:



In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :



  • outer find: find all subdirs matching DIR_MATCH_TEXT in DIR1

  • inner find: for each matching subdir found in the outer loop (call it DIR2), execute the command
    find DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt

The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.



By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.



From replies so far I gather the outer find would have to use a shell call as the argument to -exec?










share|improve this question















I'm using find with the -exec option in console, on FreeBSD (for example, find . -exec sha1 ;).



  1. How do I correctly place (and if needed, escape) redirect and other execution control symbols that could syntactically apply to either of the exec command or the find command, such as > >> | and &, where I might want the symbol to apply to the find or the exec commands on different occasions?


  2. On the same theme, what is the syntax if I want to use tee to view the output of the -exec commands on the console (to watch progress) and also appended to a file (for later use)?


(I know shells vary but hopefully the most used shells such as sh/csh are quite similar?)



Update:



In part, I want to learn if nested find -exec can be done without scripting, as a simple command. Here is the use-case that prompted the questio :



  • outer find: find all subdirs matching DIR_MATCH_TEXT in DIR1

  • inner find: for each matching subdir found in the outer loop (call it DIR2), execute the command
    find DIR2 -name "FILE_MATCH_TEXT" -exec sha1 ; >> DIR1/DIR2_hashes.txt

The aim being to create a set of files, one for each matching subdir, containing the output of some find -exec action on that subdir.



By this I mean that, if /backups contains /jan2017 and /feb2017, the result will be two files at /backups/jan2017_hashes.txt and /backups/feb2017_hashes.txt, with jan2017_hashes.txt containing the output from sha1 for (say) all .pdf files in /backups/jan2017, and feb2017_hashes.txt containing the output from sha1 for all .pdf files in /backups/feb2017.



From replies so far I gather the outer find would have to use a shell call as the argument to -exec?







shell find freebsd io-redirection






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Oct 6 '17 at 7:56

























asked Oct 3 '17 at 15:13









Stilez

441211




441211







  • 1




    Does this help? unix.stackexchange.com/questions/389705/…
    – Kusalananda
    Oct 3 '17 at 15:17










  • Duplicate question: unix.stackexchange.com/questions/111520/… . You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both stdout AND stderr output to the specified location for redirection.
    – ILMostro_7
    Oct 3 '17 at 15:18











  • These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
    – Stilez
    Oct 3 '17 at 15:25







  • 1




    @Stilez In the second case, you may move the redirection to the end of the command. It is not part of -exec nor of the options to find. See unix.stackexchange.com/questions/356118/…
    – Kusalananda
    Oct 3 '17 at 15:49












  • 1




    Does this help? unix.stackexchange.com/questions/389705/…
    – Kusalananda
    Oct 3 '17 at 15:17










  • Duplicate question: unix.stackexchange.com/questions/111520/… . You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both stdout AND stderr output to the specified location for redirection.
    – ILMostro_7
    Oct 3 '17 at 15:18











  • These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
    – Stilez
    Oct 3 '17 at 15:25







  • 1




    @Stilez In the second case, you may move the redirection to the end of the command. It is not part of -exec nor of the options to find. See unix.stackexchange.com/questions/356118/…
    – Kusalananda
    Oct 3 '17 at 15:49







1




1




Does this help? unix.stackexchange.com/questions/389705/…
– Kusalananda
Oct 3 '17 at 15:17




Does this help? unix.stackexchange.com/questions/389705/…
– Kusalananda
Oct 3 '17 at 15:17












Duplicate question: unix.stackexchange.com/questions/111520/… . You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both stdout AND stderr output to the specified location for redirection.
– ILMostro_7
Oct 3 '17 at 15:18





Duplicate question: unix.stackexchange.com/questions/111520/… . You can just do that at the end. If, for some reason, that doesn't correspond to what you're expecting, then try adding both stdout AND stderr output to the specified location for redirection.
– ILMostro_7
Oct 3 '17 at 15:18













These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
– Stilez
Oct 3 '17 at 15:25





These seem to conflict, which is what confuses me. The first suggests that redirect just doesn't work inside exec ("A shell command can include redirection and piping; -exec cannot (although the whole find can be redirected)"), the second says it works anywhere inside or out ("set the output inside the exec " and "-exec ls >> /tmp/test.log ;"). That's part of what confuses me.
– Stilez
Oct 3 '17 at 15:25





1




1




@Stilez In the second case, you may move the redirection to the end of the command. It is not part of -exec nor of the options to find. See unix.stackexchange.com/questions/356118/…
– Kusalananda
Oct 3 '17 at 15:49




@Stilez In the second case, you may move the redirection to the end of the command. It is not part of -exec nor of the options to find. See unix.stackexchange.com/questions/356118/…
– Kusalananda
Oct 3 '17 at 15:49










1 Answer
1






active

oldest

votes

















up vote
2
down vote



accepted










I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.



You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:



find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;


That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.



Pipes will work as well, with the same caveat of "one exec per file".



find . -type f -exec bash -c "ls -l $0 | grep something" ;


All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.






share|improve this answer




















  • Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
    – Stilez
    Oct 6 '17 at 7:32










  • Updated question to explain the use-case. Is it possible?
    – Stilez
    Oct 6 '17 at 7:57










Your Answer







StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f395872%2fcorrect-location-for-piping-and-redirecting-output-in-find-exec%23new-answer', 'question_page');

);

Post as a guest






























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
2
down vote



accepted










I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.



You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:



find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;


That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.



Pipes will work as well, with the same caveat of "one exec per file".



find . -type f -exec bash -c "ls -l $0 | grep something" ;


All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.






share|improve this answer




















  • Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
    – Stilez
    Oct 6 '17 at 7:32










  • Updated question to explain the use-case. Is it possible?
    – Stilez
    Oct 6 '17 at 7:57














up vote
2
down vote



accepted










I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.



You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:



find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;


That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.



Pipes will work as well, with the same caveat of "one exec per file".



find . -type f -exec bash -c "ls -l $0 | grep something" ;


All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.






share|improve this answer




















  • Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
    – Stilez
    Oct 6 '17 at 7:32










  • Updated question to explain the use-case. Is it possible?
    – Stilez
    Oct 6 '17 at 7:57












up vote
2
down vote



accepted







up vote
2
down vote



accepted






I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.



You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:



find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;


That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.



Pipes will work as well, with the same caveat of "one exec per file".



find . -type f -exec bash -c "ls -l $0 | grep something" ;


All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.






share|improve this answer












I/O redirection and piping are functions of shells (e.g., bash). Looking at the source code of the find command (https://www.gnu.org/software/findutils/), the -exec option forks and directly execs the given command, it does not send it though any shell. Given that, without help, you cannot include piping and redirection in the argument to find.



You can, however, have find invoke a shell and have that shell in turn invoke your command. When running the same through a shell, you have access to the I/O redirection operations exposed by the shell. Consider, for example:



find . -type f -exec bash -c 'ls -l $0 >> /tmp/result' ;


That will run ls -l on each file found by find (the shell refers to that file in this context as $0) and redirect the output to /tmp/result. Note that I used >> instead of >. Here, find is invoking a shell for each file, so > will result in each result overwriting the previous -- you'll end up with a file that contains only the output from the last command.



Pipes will work as well, with the same caveat of "one exec per file".



find . -type f -exec bash -c "ls -l $0 | grep something" ;


All that said, it's not immediately clear to me why you'd want to apply redirection within the context of the -exec, or what it would buy you over performing the redirection on the output of find itself.







share|improve this answer












share|improve this answer



share|improve this answer










answered Oct 6 '17 at 4:50









Andy Dalton

4,7991520




4,7991520











  • Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
    – Stilez
    Oct 6 '17 at 7:32










  • Updated question to explain the use-case. Is it possible?
    – Stilez
    Oct 6 '17 at 7:57
















  • Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
    – Stilez
    Oct 6 '17 at 7:32










  • Updated question to explain the use-case. Is it possible?
    – Stilez
    Oct 6 '17 at 7:57















Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
– Stilez
Oct 6 '17 at 7:32




Thank you, this was very helpful. The question you pose at the end is also good, its making me think, too!
– Stilez
Oct 6 '17 at 7:32












Updated question to explain the use-case. Is it possible?
– Stilez
Oct 6 '17 at 7:57




Updated question to explain the use-case. Is it possible?
– Stilez
Oct 6 '17 at 7:57

















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f395872%2fcorrect-location-for-piping-and-redirecting-output-in-find-exec%23new-answer', 'question_page');

);

Post as a guest