rsync to USB flash drive always transferring all data
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
When rsync
ing a directory to a freshly plugged-in external USB flash drive, via
rsync -av /source/ /dest/
all files get transferred (i.e. rewritten) despite no changes in the files.
Note that overwriting the files only takes place once the USB is un- and replugged. Doing the rsync
command twice in a row without unplugging the drive in-between does successfully skip the whole directory contents.
Including the -u
update option and explicitly adding the -t
option did not change anything.
The mount point remains the same (i.e. /media/user/<UUID>
, the drive is automouted by xfce
, the /dev/sdxy
obviously changes)
The hard drive source is ext4
, while the USB is vfat
with utf8
character encoding.
What could be the reason for this behaviour is it the change in the /dev/
name entry? How can I make rsync
run with properly recognizing file changes? My backup should just take seconds without this, while it now is always minutes due to the large amount of data being overwritten repeatedly, nor is the massive writing the best for the flash drive's life time expectancy.
mount rsync ext4 vfat
add a comment |
up vote
1
down vote
favorite
When rsync
ing a directory to a freshly plugged-in external USB flash drive, via
rsync -av /source/ /dest/
all files get transferred (i.e. rewritten) despite no changes in the files.
Note that overwriting the files only takes place once the USB is un- and replugged. Doing the rsync
command twice in a row without unplugging the drive in-between does successfully skip the whole directory contents.
Including the -u
update option and explicitly adding the -t
option did not change anything.
The mount point remains the same (i.e. /media/user/<UUID>
, the drive is automouted by xfce
, the /dev/sdxy
obviously changes)
The hard drive source is ext4
, while the USB is vfat
with utf8
character encoding.
What could be the reason for this behaviour is it the change in the /dev/
name entry? How can I make rsync
run with properly recognizing file changes? My backup should just take seconds without this, while it now is always minutes due to the large amount of data being overwritten repeatedly, nor is the massive writing the best for the flash drive's life time expectancy.
mount rsync ext4 vfat
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
When rsync
ing a directory to a freshly plugged-in external USB flash drive, via
rsync -av /source/ /dest/
all files get transferred (i.e. rewritten) despite no changes in the files.
Note that overwriting the files only takes place once the USB is un- and replugged. Doing the rsync
command twice in a row without unplugging the drive in-between does successfully skip the whole directory contents.
Including the -u
update option and explicitly adding the -t
option did not change anything.
The mount point remains the same (i.e. /media/user/<UUID>
, the drive is automouted by xfce
, the /dev/sdxy
obviously changes)
The hard drive source is ext4
, while the USB is vfat
with utf8
character encoding.
What could be the reason for this behaviour is it the change in the /dev/
name entry? How can I make rsync
run with properly recognizing file changes? My backup should just take seconds without this, while it now is always minutes due to the large amount of data being overwritten repeatedly, nor is the massive writing the best for the flash drive's life time expectancy.
mount rsync ext4 vfat
When rsync
ing a directory to a freshly plugged-in external USB flash drive, via
rsync -av /source/ /dest/
all files get transferred (i.e. rewritten) despite no changes in the files.
Note that overwriting the files only takes place once the USB is un- and replugged. Doing the rsync
command twice in a row without unplugging the drive in-between does successfully skip the whole directory contents.
Including the -u
update option and explicitly adding the -t
option did not change anything.
The mount point remains the same (i.e. /media/user/<UUID>
, the drive is automouted by xfce
, the /dev/sdxy
obviously changes)
The hard drive source is ext4
, while the USB is vfat
with utf8
character encoding.
What could be the reason for this behaviour is it the change in the /dev/
name entry? How can I make rsync
run with properly recognizing file changes? My backup should just take seconds without this, while it now is always minutes due to the large amount of data being overwritten repeatedly, nor is the massive writing the best for the flash drive's life time expectancy.
mount rsync ext4 vfat
mount rsync ext4 vfat
asked Mar 2 '16 at 23:00
Fiximan
3,123524
3,123524
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
up vote
6
down vote
accepted
Your FAT drive can store timestamps only to two second accuracy. When you unplug and replug the drive you effectively break all the file times. See the --modify-window
option for a workaround.
Secondly, you're never going to see fast backups with rsync
like this, because when copying locally it behaves much like cp
.
Setting a value for--modify-window
solved the issue, thank you.
– Fiximan
Mar 4 '16 at 21:35
add a comment |
up vote
1
down vote
By default, rsync identifies changed files through their modification time and file size. A quick Google search for "vfat timestamp" reveals several problems related to the tz=UTC
mount option.
Could it be that the timestamps on external drive differ from the internal drive and rsync therefore always considers the files to be different? If that is the case, you should probably check your mount options.
Alternatively (and actually regardless of whether my theory is right), you could try to run rsync with the -c
option, which makes it check for changes through hashes of the file contents. This potentially slows down things, but I'm unsure about the practical impact.
add a comment |
up vote
0
down vote
My Environment
Raspberry Pi3 B+,Raspbian Strech(Linux)
I added next 1 line below into /etc/fstabdev/sda1 /media/pi/TOSHIBA vfat defaults 0 0
Thensudo umount /media/pi/TOSHIBA
To find device name you can use next command lsusb
Then I re-inserted the USB MEMORY.
Then USB MEMORY's owner was changed like next 1 line below.drwxr-xr-x 3 root root 8192 11月 9 23:22 TOSHIBA
Then I have been able to backup with permission,owner and timestamps using next 1 line below.rsync -alvptgo /home/pi /media/pi/TOSHIBA/pi
which I have really used as DIFF backup command.
But PAY ATTENTION to that you must insert a USB MEMORY in the slot when you reboot
OR
Delete that 1 line in /etc/fstab before you reboot.
New contributor
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
6
down vote
accepted
Your FAT drive can store timestamps only to two second accuracy. When you unplug and replug the drive you effectively break all the file times. See the --modify-window
option for a workaround.
Secondly, you're never going to see fast backups with rsync
like this, because when copying locally it behaves much like cp
.
Setting a value for--modify-window
solved the issue, thank you.
– Fiximan
Mar 4 '16 at 21:35
add a comment |
up vote
6
down vote
accepted
Your FAT drive can store timestamps only to two second accuracy. When you unplug and replug the drive you effectively break all the file times. See the --modify-window
option for a workaround.
Secondly, you're never going to see fast backups with rsync
like this, because when copying locally it behaves much like cp
.
Setting a value for--modify-window
solved the issue, thank you.
– Fiximan
Mar 4 '16 at 21:35
add a comment |
up vote
6
down vote
accepted
up vote
6
down vote
accepted
Your FAT drive can store timestamps only to two second accuracy. When you unplug and replug the drive you effectively break all the file times. See the --modify-window
option for a workaround.
Secondly, you're never going to see fast backups with rsync
like this, because when copying locally it behaves much like cp
.
Your FAT drive can store timestamps only to two second accuracy. When you unplug and replug the drive you effectively break all the file times. See the --modify-window
option for a workaround.
Secondly, you're never going to see fast backups with rsync
like this, because when copying locally it behaves much like cp
.
answered Mar 2 '16 at 23:19
roaima
41.8k550114
41.8k550114
Setting a value for--modify-window
solved the issue, thank you.
– Fiximan
Mar 4 '16 at 21:35
add a comment |
Setting a value for--modify-window
solved the issue, thank you.
– Fiximan
Mar 4 '16 at 21:35
Setting a value for
--modify-window
solved the issue, thank you.– Fiximan
Mar 4 '16 at 21:35
Setting a value for
--modify-window
solved the issue, thank you.– Fiximan
Mar 4 '16 at 21:35
add a comment |
up vote
1
down vote
By default, rsync identifies changed files through their modification time and file size. A quick Google search for "vfat timestamp" reveals several problems related to the tz=UTC
mount option.
Could it be that the timestamps on external drive differ from the internal drive and rsync therefore always considers the files to be different? If that is the case, you should probably check your mount options.
Alternatively (and actually regardless of whether my theory is right), you could try to run rsync with the -c
option, which makes it check for changes through hashes of the file contents. This potentially slows down things, but I'm unsure about the practical impact.
add a comment |
up vote
1
down vote
By default, rsync identifies changed files through their modification time and file size. A quick Google search for "vfat timestamp" reveals several problems related to the tz=UTC
mount option.
Could it be that the timestamps on external drive differ from the internal drive and rsync therefore always considers the files to be different? If that is the case, you should probably check your mount options.
Alternatively (and actually regardless of whether my theory is right), you could try to run rsync with the -c
option, which makes it check for changes through hashes of the file contents. This potentially slows down things, but I'm unsure about the practical impact.
add a comment |
up vote
1
down vote
up vote
1
down vote
By default, rsync identifies changed files through their modification time and file size. A quick Google search for "vfat timestamp" reveals several problems related to the tz=UTC
mount option.
Could it be that the timestamps on external drive differ from the internal drive and rsync therefore always considers the files to be different? If that is the case, you should probably check your mount options.
Alternatively (and actually regardless of whether my theory is right), you could try to run rsync with the -c
option, which makes it check for changes through hashes of the file contents. This potentially slows down things, but I'm unsure about the practical impact.
By default, rsync identifies changed files through their modification time and file size. A quick Google search for "vfat timestamp" reveals several problems related to the tz=UTC
mount option.
Could it be that the timestamps on external drive differ from the internal drive and rsync therefore always considers the files to be different? If that is the case, you should probably check your mount options.
Alternatively (and actually regardless of whether my theory is right), you could try to run rsync with the -c
option, which makes it check for changes through hashes of the file contents. This potentially slows down things, but I'm unsure about the practical impact.
edited Mar 2 '16 at 23:29
answered Mar 2 '16 at 23:18
F30
216112
216112
add a comment |
add a comment |
up vote
0
down vote
My Environment
Raspberry Pi3 B+,Raspbian Strech(Linux)
I added next 1 line below into /etc/fstabdev/sda1 /media/pi/TOSHIBA vfat defaults 0 0
Thensudo umount /media/pi/TOSHIBA
To find device name you can use next command lsusb
Then I re-inserted the USB MEMORY.
Then USB MEMORY's owner was changed like next 1 line below.drwxr-xr-x 3 root root 8192 11月 9 23:22 TOSHIBA
Then I have been able to backup with permission,owner and timestamps using next 1 line below.rsync -alvptgo /home/pi /media/pi/TOSHIBA/pi
which I have really used as DIFF backup command.
But PAY ATTENTION to that you must insert a USB MEMORY in the slot when you reboot
OR
Delete that 1 line in /etc/fstab before you reboot.
New contributor
add a comment |
up vote
0
down vote
My Environment
Raspberry Pi3 B+,Raspbian Strech(Linux)
I added next 1 line below into /etc/fstabdev/sda1 /media/pi/TOSHIBA vfat defaults 0 0
Thensudo umount /media/pi/TOSHIBA
To find device name you can use next command lsusb
Then I re-inserted the USB MEMORY.
Then USB MEMORY's owner was changed like next 1 line below.drwxr-xr-x 3 root root 8192 11月 9 23:22 TOSHIBA
Then I have been able to backup with permission,owner and timestamps using next 1 line below.rsync -alvptgo /home/pi /media/pi/TOSHIBA/pi
which I have really used as DIFF backup command.
But PAY ATTENTION to that you must insert a USB MEMORY in the slot when you reboot
OR
Delete that 1 line in /etc/fstab before you reboot.
New contributor
add a comment |
up vote
0
down vote
up vote
0
down vote
My Environment
Raspberry Pi3 B+,Raspbian Strech(Linux)
I added next 1 line below into /etc/fstabdev/sda1 /media/pi/TOSHIBA vfat defaults 0 0
Thensudo umount /media/pi/TOSHIBA
To find device name you can use next command lsusb
Then I re-inserted the USB MEMORY.
Then USB MEMORY's owner was changed like next 1 line below.drwxr-xr-x 3 root root 8192 11月 9 23:22 TOSHIBA
Then I have been able to backup with permission,owner and timestamps using next 1 line below.rsync -alvptgo /home/pi /media/pi/TOSHIBA/pi
which I have really used as DIFF backup command.
But PAY ATTENTION to that you must insert a USB MEMORY in the slot when you reboot
OR
Delete that 1 line in /etc/fstab before you reboot.
New contributor
My Environment
Raspberry Pi3 B+,Raspbian Strech(Linux)
I added next 1 line below into /etc/fstabdev/sda1 /media/pi/TOSHIBA vfat defaults 0 0
Thensudo umount /media/pi/TOSHIBA
To find device name you can use next command lsusb
Then I re-inserted the USB MEMORY.
Then USB MEMORY's owner was changed like next 1 line below.drwxr-xr-x 3 root root 8192 11月 9 23:22 TOSHIBA
Then I have been able to backup with permission,owner and timestamps using next 1 line below.rsync -alvptgo /home/pi /media/pi/TOSHIBA/pi
which I have really used as DIFF backup command.
But PAY ATTENTION to that you must insert a USB MEMORY in the slot when you reboot
OR
Delete that 1 line in /etc/fstab before you reboot.
New contributor
New contributor
answered 4 hours ago
USAKU
1
1
New contributor
New contributor
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f267204%2frsync-to-usb-flash-drive-always-transferring-all-data%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password