mjhjmtu

I just finished reading an article titled "Why you shouldn't parse the output of ls". I want to loop each file in a directory, and the article states that the following is one of the most basic and ideal ways.

for f in *; do
 echo $f
 # …do stuff
done

The one thing I do not understand is how can I ignore certain files and directories? I use ZSH and have turned extendedglob on. Because of this, I thought this would be viable: for f in *^.jpg; do…, but it only echoes "*^.jpg", no files.

How can I keep the loop mostly as is, but incorporate the ability to exclude certain patterns? Bonus question: can you make the glob recursive?

Zsh provides very nice ways to enumerate files. They're documented in the manual under “Filename Generation”, but the zsh manual isn't very easy to follow.

By default, if there are no matching files, you'll get an error, which is often desirable on the command line, but not in scripts. To disable this error, either put this near the beginning of your script to turn on the null_glob option:

setopt null_glob
…
for x in *; do …

or use the N glob qualifier:

# Reliably iterate over non-dot files
for x in *(N); do …

Either snippet above iterates over all the files in the current directory that are not dot files. If there is no such file, the loop doesn't run at all (unlike sh where the loop body unhelpfully runs once on the unexpanded glob *, and default zsh where the glob causes an error).

To include dot files in the iteration, either turn on the got_glob option or use the D glob qualifier.

# Reliably iterate over all files in the current directory
for x in *(DN); do …

I recommend the glob qualifier method, even if it's more verbose, because it will keep behaving the same way even if you copy-paste that snippet to a script that didn't turn on the same options.

To exclude certain files based on their name, you can use the glob operators ^ and ~. Note that these require setopt extended_glob in your script² (this is not inherited from your runtime environment or your .zshrc¹). For example, to exclude *.jpg files, either write *~*.jpg or ^*.jpg.

(Note that what you wrote, *^.jpg, would by default cause an error in zsh. If you're seeing *^.jpg printed out, then either you're running zsh with nomatch turned off (it's on by default), which is normally only done when emulating sh or ksh, or else you're actually running that script under an sh shell, probably because it's missing a shebang line at the top.)

setopt extended_glob
# Iterate over all the files in the current directory except *.jpg and .*
for x in *~*.jpg(N); do …

If you want to traverse subdirectories recursively, it's easy: just use **/.

setopt extended_glob
# Iterate over all the files in the current directory and its (grand-)*children except *.jpg and .*
for x in **/*~*.jpg(N); do …

Through glob qualifiers, you can also select files based on conditions other than their name. For example, to loop over regular files only, excluding directories, symbolic links, etc.:

# Iterate over all regular files in the current directory and its (grand-)*children except *.jpg and .*
for x in **/*(.DN); do …

You can even run arbitrary code to decide whether to include a file or not with the e or + glob qualifier. However, when the wildcard pattern is used in a for loop, it's clearer to put the filtering code at the top of the loop body.

¹ _{You could put setopt extended_glob in ~/.zshenv, but I strongly recommend against using .zshenv, because any script that makes assumptions about what's in there would break on a different machine or a different account.}

² _{If you're writing a zsh completion function, these run in an environment where extended_glob is turned on.}