mjhjmtu

I need to exclude a bunch of paths from a find command. For example:

find "$(pwd)" -not ( 
 -path "*/.git"
 -o -path "*/.git/*"
 -o -path "*/.vscode"
 -o -path "*/.vscode/*"
 -o -path "*/node_modules"
 -o -path "*/node_modules/*"
 -o -path "*/Image"
 -o -path "*/Image/*"
 -o -path "*/Rendered"
 -o -path "*/Rendered/*"
 -o -path "*/iNotebook"
 -o -path "*/iNotebook/*"
 -o -path "*/GeneratedTest"
 -o -path "*/GeneratedTest/*"
 -o -path "*/GeneratedOutput"
 -o -path "*/GeneratedOutput/*"
 -o -path "*/*_files" ) -type d

However, I want to read these paths from a text file instead of listing them all on the command-line. How can I do that?

Construct an array that you later use in your call to find. The following script reads the newline-delimited path patterns from its standard input and calls find:

#!/bin/sh

set --

while IFS= read -r path; do
 set -- "$@" -o -path "$path"
done

shift # remove initial "-o" from $@

find . -type d ! '(' "$@" ')'

You would run this with

./script.sh <paths.txt

where paths.txt might look like

*/.git
*/.git/*
*/.vscode
*/.vscode/*
*/node_modules
*/node_modules/*
*/Image
*/Image/*
*/Rendered
*/Rendered/*
*/iNotebook
*/iNotebook/*
*/GeneratedTest
*/GeneratedTest/*
*/GeneratedOutput
*/GeneratedOutput/*
*/*_files

Or, since your path patterns are all basically names of directories:

#!/bin/sh

set --

while IFS= read -r dirname; do
 set -- "$@" -o '(' -name "$dirname" -prune ')'
done

shift # remove initial "-o" from $@

find . -type d ! '(' "$@" ')'

with the pattern file containing

.git
.vscode
node_modules
Image
Rendered
iNotebook
GeneratedTest
GeneratedOutput
*_files

This variant of the code would stop find from even descending into the directories matching the patterns in the file, whereas the first script (as well as your code) would test the -path patterns against everything in the excluded directories regardless of the fact that you're not interested in anything below those paths.

You can use grep and find’s -exec to filter files against a list of paths, either as regular expressions or fixed strings. Adapting your example, create a file named paths containing

/.git$
/.git/
/.vscode$
/.vscode/
/node_modules$
/node_modules/
/Image$
/Image/
/Rendered$
/Rendered/
/iNotebook$
/iNotebook/
/GeneratedTest$
/GeneratedTest/
/GeneratedOutput$
/GeneratedOutput/
/.*_files$

Then run

find /your/search/path -type d ! -exec sh -c "echo | grep -q -f paths" ; -print

This looks for directories under /your/search/path, and for each one it finds, uses grep to determine whether it matches a pattern in paths; if it doesn’t, it prints it. This is intended as a base for extension; if you only care about the paths of directories which don’t match the patterns in a file, and none of the paths cover multiple lines, you can post-process the output using a single grep invocation:

find /your/search/path -type d | grep -v -f paths

If you’re really not interested in certain paths at all (i.e. your patterns always match a directory name and then everything under that directory), you could make things simpler by pruning:

find /your/search/path -type d ( -exec sh -c "echo | grep -q -f paths" ; -prune -o -print )

with the following contents in paths:

/.git$
/.vscode$
/node_modules$
/Image$
/Rendered$
/iNotebook$
/GeneratedTest$
/GeneratedOutput$
/.*_files$

What can be done is to built up the command using awk and pass it to find as variable in a "wrapper" script or shell function

p=$( awk 'printf "-not -path %s ",$0' "$1" )
find "$PWD" $p -type d

And call it as ./find_wrapper.sh paths.txt, where path.txt is the list of quoted paths.

'*/.git'
'*/.git/*'
'*/.vscode'
'*/.vscode/*'
'*/node_modules'
'*/node_modules/*'
'*/Image'
...

Why it's done this way ? The reason awk builts a one whole line is because there's no reason to do that in script - the line continuations are for making the command look more organized, but functionally it doesn't give any advantage. $p is unquoted, since we actually want word splitting here. Otherwise find sees it as one giant string and not individual flags and arguments. As for single quotes, that's to avoid glob effect in double quotes.

Alternatively as pipeline

awk 'printf "-not -path %s ",$0' "$1" | xargs -L 1 find "$PWD" -type d