mjhjmtu

I am trying to use rsync to copy a specific set of subdirectories to back them up. My directory structure looks something like this:

/path/to/data/foo1
/path/to/data/foo2
/path/to/data/bar1
/path/to/data/bar2

I just want to copy the 'foo' folders, ignoring the 'bar' ones.

I also want to ignore any .zip files inside those folders.

I have tried many combinations of arguments in rsync, but can't get it to do exactly what I want.

I can do the transfer as I want to using two commands:

rsync --dry-run -a --human-readable --stats --progress --exclude"*.zip" "/path/to/data/foo1" "/mnt/backup/drive/"
rsync --dry-run -a --human-readable --stats --progress --exclude"*.zip" "/path/to/data/foo2" "/mnt/backup/drive"

Form reading elsewhere, I expect something like this to work

rsync --dry-run -a --human-readable --stats --progress --include="data/foo*" --exclude="*.zip" --exclude="*" "/path/to/data" "/mnt/backup/drive"

But this also seems to capture the 'bar' folders.

The ordering of the --include and --exclude arguments is confusing me. As is the need to specify the 'data' folder in the --include argument is not the functionality I would expect.

What is the obvious flag that I'm missing!

Thanks for your input.

Ben

It will work for you to run rsync in this way:

rsync --dry-run -av --human-readable --stats --progress --exclude="*.zip" /path/to/data/foo* /path/to/copy/to/

perhaps you'd like to shorter the command, in this way:

rsync -avhn --stats --progress --exclude="*.zip" /path/to/data/foo* /path/to/copy/to/

quoting from man rsync

-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)

-v, --verbose increase verbosity

-h, --human-readable output numbers in a human-readable format

-n, --dry-run perform a trial run with no changes made

when you are ready run the command without -n

it's also possible to save a logfile with this argument :

--log-file=

so the rsync command above will become:

rsync -avhn --stats --progress --exclude="*.zip" --log-file=logfile.log /path/to/data/foo* /path/to/copy/to/

the logfile will be written in the same directory where you run the command, if you need to save it in another location, you must use an absolute path.
for example:

--log-file=/path/to/logfile

You've already got a suitable answer, and that's fine. I wanted to address the other part of your question that asks why the --include functionality doesn't do what you expect.

First of all, let's look at how the --include and --exclude filters operate.

• 
  aaa will apply to any file or directory named aaa
  


• 
  /aaa will apply to any file or directory named aaa at the top of the source path


• 
  aaa/ will apply to any directory called aaa
  

You can combine these, so a path of bbb/ccc/ indicates any directory ccc that is a child of a directory bbb anywhere in the source tree, but /bbb/ccc/ is a directory ccc inside a directory bbb that is tied to the top of the source tree.

Include and exclude operations are processed from left to right (the first operation is more important that the second, and the second is more important than the third).

Now let's look at your specific example:

rsync -ah --include="data/foo*" --exclude="*.zip" --exclude="*" "/path/to/data" "/mnt/backup/drive"

The filter rules say, in order:

1. Include all files or directories whose name begins with foo that are in a directory called data somewhere below the source path.


2. Exclude all *.zip files (or directories).


3. Exclude everything we haven't already mentioned.

The first include would match /path/to/data/data/foo1 but it would not match /path/to/data/foo1. I suspect this is the primary misunderstanding.

You can match your /foo* directories with a very similar solution:

rsync -ah --include="/foo*/" --exclude="*.zip" --exclude="*" "/path/to/data/" "/mnt/backup/drive"

But actually, if I were writing this I would probably include the /foo* directories in the source path specification (notice the foo* is unquoted so the shell can get at the wildcard) and only worry about excluding the unwanted zip files:

rsync -ah --exclude="*.zip" /path/to/data/foo*/ /mnt/backup/drive