mjhjmtu

I'm testing a script to do a by-directional sync of two directories intelligently using rsync.

Since I'm testing many of the rsync options are not applicable to the testing environment including the lines:

personal_excludes="
--exclude='home/jesse/scripts/'
--exclude='home/jesse/.*/'"

So when I thought I should have -prune on the end of that exclude I really didn't think it would make any difference in my testing environment.

However, adding -prune to these two lines causes rsync to do a DRY-RUN!!?? What The ??

Anyone care to enlighten me on how/why this is? Should I file a bug report for rsync?

Working Directory

jesse@Limbo ~/dev/sync-script-testing $ ls -la . *
-rwxr-x--- 1 jesse jesse 1910 Dec 29 14:04 test-script.sh

.:
total 24
drwxr-x--- 5 jesse jesse 4096 Dec 29 14:05 .
drwxrwxr-x 7 jesse jesse 4096 Dec 29 13:53 ..
drwxr-x--- 2 jesse jesse 4096 Dec 29 13:43 dir1
drwxr-x--- 2 jesse jesse 4096 Dec 29 13:37 dir2
drwxr-x--- 8 jesse jesse 4096 Dec 29 14:03 .git
-rwxr-x--- 1 jesse jesse 1910 Dec 29 14:04 test-script.sh

dir1:
total 12
drwxr-x--- 2 jesse jesse 4096 Dec 29 13:43 .
drwxr-x--- 5 jesse jesse 4096 Dec 29 14:05 ..
-rw-r----- 1 jesse jesse 0 Dec 29 13:33 file2
-rw-r----- 1 jesse jesse 0 Dec 29 13:34 file3
-rw-r----- 1 jesse jesse 0 Dec 29 13:35 file4
-rw-r----- 1 jesse jesse 32 Dec 29 14:04 .lastsync

dir2:
total 12
drwxr-x--- 2 jesse jesse 4096 Dec 29 13:37 .
drwxr-x--- 5 jesse jesse 4096 Dec 29 14:05 ..
-rw-r----- 1 jesse jesse 0 Dec 29 13:33 file1
-rw-r----- 1 jesse jesse 0 Dec 29 13:34 file3
-rw-r----- 1 jesse jesse 0 Dec 29 13:35 file4
-rw-r----- 1 jesse jesse 32 Dec 29 14:04 .lastsync

Script Contents

jesse@Limbo ~/dev/sync-script-testing $ cat test-script.sh 
#!/usr/bin/bash

#rsync -av --files-from=<(cd dir1 && find ./ -newermt "$(sed 's/^Successful sync on //' sync.log)") --exclude=/sync.log ./dir1/ ./dir2/ && echo "Successful sync on $(date -R)" | tee dir2/sync.log > dir1/sync.log
#
# # Perform the Sync
# echo -e "n[1] Uppdate HDD -> USB DO NOT Include configuration .dot file - They take too longn"

# First do a sync of ONLY those files modified since last run WITHOUT deleting any files

rsync_general_args="
 --verbose
 --human-readable
 --progress
 --recursive
 --update
 --links
 --perms
 --times
 --group
 --owner
 --devices
 --specials
 --hard-links
 --xattrs
 --one-file-system
 --one-file-system" # specifying --one-file-system twice means something different!

personal_excludes="
 --exclude='home/jesse/scripts/'
 --exclude='home/jesse/.*/'"

# Build list of files to sync
# First use mktemp to create a randomly named temporary directory to keep tempfiles in. This prevents hackers from intercepting data
tmpdir=$(mktemp -d "$TMPDIR:-/tmp/$(basename $0).XXXXXXXXXXXX")

# Find files modified since the last sync, date saved in .lastcync
# Do this in a subshell so as not to change the current directory
$(cd dir1 && find ./ -newermt "$(cat .lastsync)" ! -path "./.lastsync" >$tmpdir/rsync_files_to_sync_from_dir1)
$(cd dir2 && find ./ -newermt "$(cat .lastsync)" ! -path "./.lastsync" >$tmpdir/rsync_files_to_sync_from_dir2)

echo; echo --files-from dir1=$(cat $tmpdir/rsync_files_to_sync_from_dir1)
rsync $rsync_general_args $personal_excludes --files-from=$tmpdir/rsync_files_to_sync_from_dir1 --exclude=/.lastsync ./dir1/ ./dir2/ && date -R > ./dir1/.lastsync

echo; echo --files-from dir2=$(cat $tmpdir/rsync_files_to_sync_from_dir2)
rsync $rsync_general_args $personal_excludes --files-from=$tmpdir/rsync_files_to_sync_from_dir2 --exclude=/.lastsync ./dir2/ ./dir1/ && date -R > ./dir2/.lastsync


# Clean Up
rm -r $tmpdir

Script output

jesse@Limbo ~/dev/sync-script-testing $ ./test-script.sh 

--files-from dir1=
sending incremental file list

sent 18 bytes received 12 bytes 60.00 bytes/sec
total size is 0 speedup is 0.00

--files-from dir2=
sending incremental file list

sent 18 bytes received 12 bytes 60.00 bytes/sec
total size is 0 speedup is 0.00

Make a change to the script: Adding "-prune"

jesse@Limbo ~/dev/sync-script-testing $ git log -p --grep WHY??
commit d9d50bdd289616faaf3d174e6023ba16a5286b53
Author: Jesse the Wind Wanderer <webmaster@windwanderer.com.au>
Date: Sat Dec 29 14:03:43 2018 +0800

 adding '-prune' causes rsync to do DRY-RUN. WHY???

diff --git a/test-script.sh b/test-script.sh
index 37ee2e6..b45f4a0 100755
--- a/test-script.sh
+++ b/test-script.sh
@@ -26,8 +26,8 @@ rsync_general_args="
 --one-file-system" # specifying --one-file-system twice means something different!

 personal_excludes="
- --exclude='home/jesse/scripts/'
- --exclude='home/jesse/.*/'"
+ --exclude='home/jesse/scripts/' -prune
+ --exclude='home/jesse/.*/' -prune"

 # Build list of files to sync
 # First use mktemp to create a randomly named temporary directory to keep tempfiles in. This prevents hackers from intercepting data

Script now runs rsync with DRY-RUN

jesse@Limbo ~/dev/sync-script-testing $ ./test-script.sh 

--files-from dir1=
sending incremental file list
./
file2

sent 123 bytes received 22 bytes 290.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)

--files-from dir2=
sending incremental file list
./
file1

sent 127 bytes received 22 bytes 298.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)

Ohhhhhhhhh, Now I feel silly. So as a lesson to myself and others I'll post this. -prune is an option of find not rsync!! find is unusual in that it only uses single '-' with long-options.

Adding -prune to rsync meant that each of the individual letters were interpreted as a single option '-p' '-r' '-u' '-n' (which = Dry-Run!!) and '-e'