mjhjmtu

I have a folder with more than a million files that needs sorting, but I can't really do anything because mv outputs this message all the time

-bash: /bin/mv: Argument list too long

I'm using this command to move extension-less files:

mv -- !(*.jpg|*.png|*.bmp) targetdir/

xargs is the tool for the job. That, or find with -exec … +. These tools run a command several times, with as many arguments as can be passed in one go.

Both methods are easier to carry out when the variable argument list is at the end, which isn't the case here: the final argument to mv is the destination. With GNU utilities (i.e. on non-embedded Linux or Cygwin), the -t option to mv is useful, to pass the destination first.

If the file names have no whitespace nor any of "', then you can simply provide the file names as input to xargs (the echo command is a bash builtin, so it isn't subject to the command line length limit):

echo !(*.jpg|*.png|*.bmp) | xargs mv -t targetdir

You can use the -0 option to xargs to use null-delimited input instead of the default quoted format.

printf '%s' !(*.jpg|*.png|*.bmp) | xargs -0 mv -t targetdir

Alternatively, you can generate the list of file names with find. To avoid recursing into subdirectories, use -type d -prune. Since no action is specified for the listed image files, only the other files are moved.

find . -name . -o -type d -prune -o 
 -name '*.jpg' -o -name '*.png' -o -name '*.bmp' -o 
 -exec mv -t targetdir/ +

(This includes dot files, unlike the shell wildcard methods.)

If you don't have GNU utilities, you can use an intermediate shell to get the arguments in the right order. This method works on all POSIX systems.

find . -name . -o -type d -prune -o 
 -name '*.jpg' -o -name '*.png' -o -name '*.bmp' -o 
 -exec sh -c 'mv "$@" "$0"' targetdir/ +

In zsh, you can load the mv builtin:

setopt extended_glob
zmodload zsh/files
mv -- ^*.(jpg|png|bmp) targetdir/

or if you prefer to let mv and other names keep referring to the external commands:

setopt extended_glob
zmodload -Fm zsh/files b:zf_*
zf_mv -- ^*.(jpg|png|bmp) targetdir/

or with ksh-style globs:

setopt ksh_glob
zmodload -Fm zsh/files b:zf_*
zf_mv -- !(*.jpg|*.png|*.bmp) targetdir/

Alternatively, using GNU mv and zargs:

autoload -U zargs
setopt extended_glob
zargs -- ./^*.(jpg|png|bmp) -- mv -t targetdir/

The operating system's argument passing limit does not apply to expansions which happen within the shell interpreter. So in addition to using xargs or find, we can simply use a shell loop to break up the processing into individual mv commands:

for x in *; do case "$x" in *.jpg|*.png|*.bmp) ;; *) mv -- "$x" target ;; esac ; done

This uses only POSIX Shell Command Language features and utilities. This one-liner is clearer with indentation, with unnecessary semicolons removed:

for x in *; do
 case "$x" in
 *.jpg|*.png|*.bmp) 
 ;; # nothing
 *) # catch-all case
 mv -- "$x" target
 ;;
 esac
done

For a more aggressive solution than those previously offered, pull up your kernel source and edit include/linux/binfmts.h

Increase the size of MAX_ARG_PAGES to something larger than 32. This increases the amount of memory the kernel will allow for program arguments, thereby allowing you to specify your mv or rm command for a million files or whatever you're doing. Recompile, install, reboot.

BEWARE! If you set this too large for your system memory, and then run a command with a lot of arguments BAD THINGS WILL HAPPEN! Be extremely cautious doing this to multi-user systems, it makes it easier for malicious users to use up all your memory!

If you don't know how to recompile and reinstall your kernel manually, it's probably best that you just pretend this answer doesn't exist for now.

A more simple solution using "$origin"/!(*.jpg|*.png|*.bmp) instead of a catch block:

for file in "$origin"/!(*.jpg|*.png|*.bmp); do mv -- "$file" "$destination" ; done

Thanks to @Score_Under

For a multi-line script you can do the following (notice the ; before the done is dropped):

for file in "$origin"/!(*.jpg|*.png|*.bmp); do # don't copy types *.jpg|*.png|*.bmp
 mv -- "$file" "$destination" 
done 

To do a more generalized solution that moves all files, you can do the one-liner:

for file in "$origin"/*; do mv -- "$file" "$destination" ; done

Which looks like this if you do indentation:

for file in "$origin"/*; do
 mv -- "$file" "$destination"
done 

This takes every file in the origin and moves them one by one to the destination. The quotes around $file are necessary in case there are spaces or other special characters in the filenames.

Here is an example of this method that worked perfectly

for file in "/Users/william/Pictures/export_folder_111210/"*.jpg; do
 mv -- "$file" "/Users/william/Desktop/southland/landingphotos/";
done