mjhjmtu

I want to back up my /home directory with rsync. I have read rsync's man page and decided to use filter rules for this task.

What I would like to achieve: Exclude all files and directories in the Repos directory but keep all pull_all.sh files and output directories --- regardless where they are located within the Repos directory.

So far, I have ended up with following filter list, but this backs up only the pull_all.sh files but not the output directories:

# Files prefixed with "+ " are included. Files prefixed with "- " are excluded.
#
# The order of included and excluded files matters! For instance, if a folder
# is excluded first, no subdirectory can be included anymore. Therefore,
# mention included files first. Then, mention excluded files.
#
# See section "FILTER RULES" of rsync manual for more details.


# Included Files

# TODO: This rules do not work properly!
+ output/***
+ pull_all.sh
- Repos/**

# Excluded Files

- .android
- .cache
...

I use the filter list in my script run_rsync.sh:

#!/bin/bash

date="$(date +%Y-%m-%d)"
hostname="$(hostname)"

# debug_mode="" # to disable debug mode
debug_mode="--list-only"

# Note: With trailing "/" at source directory, source directory is not created at destination.
rsync $debug_mode --archive --delete --human-readable --filter="merge $hostname.rsync.filters" --log-file=logfiles/$date-$hostname-home.log --verbose /home backup/

Unfortunately, the existing StackExchange threads have not solved my problems:

• https://stackoverflow.com/questions/8270519/rsync-exclude-a-directory-but-include-a-subdirectory


• Using Rsync include and exclude options to include directory and subdirectory but exlude files in subdirectory

What's going wrong here?

[Update] Here is an example how the home directory looks like and which files to keep and which files to ignore:

user@hostname:~$ tree /home/ | head
/home/
└── user
 ├── Desktop -> keep this
 │   ├── file1 -> keep this
 │   └── file2 -> keep this
 ├── Documents -> keep this
 ├── Repos
 │   ├── pull_all.sh -> keep this
 ├── subdir1
 │ ├── output -> keep this
 ├── subdir2
 ├── another_subdir
 ├── output -> keep this
 ├── subdir3 -> do not keep (because does not contain any "output")
 ├── file3 -> do not keep

Slightly restating what I've interpreted as your requirements,

• Include all pull_all.sh files regardless of where we find them


• Include all output directories and their contents regardless of where we find them


• Exclude the Repos directory, other than what we have already stated


• Include everything else

This can be specified as follows

rsync --dry-run --prune-empty-dirs -av

 --include 'pull_all.sh'
 --include 'Repos/**/output/***'

 --include '*/'

 --exclude 'Repos/***'

 /home backup/

Some notes

• The --include '*/' is required so that rsync will consider heading down into the Repos directory tree (to look for pull_all.sh files), which would otherwise be excluded by the final --exclude statement.


• The three different uses of * are different:
  
  
  • 
    * matches anything except / characters
  
  
  • 
    ** matches anything including / characters
  
  
  • 
    dir/*** is a shortcut equivalent to specifying dir/ and dir/**.
  
  


• The --prune-empty-dirs flag stops rsync creating empty directories, which is particularly important as we need to process the Repos directory tree looking for pull_all.sh and output items.


• Remove --dry-run when you are happy with the results.