Using dm-cache for transparent ongoing backup
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
Is there any chance to use dm-cache for ongoing SSD backup?
I want to have very fast and cheap eventually consistent failure resistant partition.
Use case:
We have regular ssd (1Gb) (cache) + hdd (1Gb) (origin) installation.
If ssd fails it should be able to take last available snapshot from hdd.
If hdd fails we have fully worked filesystem on ssd.
I dont want to use any manual snapshots and care about scheduling.
Key question is if cache size = origin size, could origin be fully restored automatically?
mdadm cache ssd block-device bcache
asked Feb 19 at 9:01
nikolobok
11
 |Â
show 2 more comments
up vote
0
down vote
favorite
Is there any chance to use dm-cache for ongoing SSD backup?
I want to have very fast and cheap eventually consistent failure resistant partition.
Use case:
We have regular ssd (1Gb) (cache) + hdd (1Gb) (origin) installation.
If ssd fails it should be able to take last available snapshot from hdd.
If hdd fails we have fully worked filesystem on ssd.
I dont want to use any manual snapshots and care about scheduling.
Key question is if cache size = origin size, could origin be fully restored automatically?
mdadm cache ssd block-device bcache
asked Feb 19 at 9:01
nikolobok
11
Why not just buy a 2nd SSD and use some form of RAID-1 aka mirroring (with mdadm, lvm, btrfs, or zfs, etc)? Drive mirroring is a tried and tested technology that Just Works. DIY with dm-cache isn't (not for this kind of task, anyway). BTW, with btrfs or zfs you can also perform regular incremental backups with,btrfs sendorzfs sendto a remote machine...and (unlike tar or rsync) they're a low-impact backup that can be performed multiple times per hour without unduly affecting overall system performance.
– cas
Feb 19 at 9:31
Because it is just 2 times more expensive
– nikolobok
Feb 19 at 9:50
Yes. there's a cost to improve the safety of your data. IMO, it's worth every cent. Personally, I'd never store anything important on anything less than a mirrored pair of drives (I use ZFS - gives me snapshots, compression, error correction, and lots of other useful stuff as well as mirroring) in addition to regular backups. If you want to do it cheaper, use 2 x HDDs and then add a smallish fast SSD for caching (e.g. with mdadm+bcache, or ZFS can use SSDs to cache HDD pools).
– cas
Feb 19 at 14:50
I want hdd raid0 + ssd (dm-cache) sensitive data on ssd + backed up by hdd
– nikolobok
Feb 20 at 14:54
if you want raid-0 then you are effectively saying "I do not care at all about my data. It is garbage and if I lost it all I wouldn't care in the slighest".
– cas
Feb 21 at 0:49
 |Â
show 2 more comments
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Is there any chance to use dm-cache for ongoing SSD backup?
I want to have very fast and cheap eventually consistent failure resistant partition.
Use case:
We have regular ssd (1Gb) (cache) + hdd (1Gb) (origin) installation.
If ssd fails it should be able to take last available snapshot from hdd.
If hdd fails we have fully worked filesystem on ssd.
I dont want to use any manual snapshots and care about scheduling.
Key question is if cache size = origin size, could origin be fully restored automatically?
mdadm cache ssd block-device bcache
asked Feb 19 at 9:01
nikolobok
11
Is there any chance to use dm-cache for ongoing SSD backup?
I want to have very fast and cheap eventually consistent failure resistant partition.
Use case:
We have regular ssd (1Gb) (cache) + hdd (1Gb) (origin) installation.
If ssd fails it should be able to take last available snapshot from hdd.
If hdd fails we have fully worked filesystem on ssd.
I dont want to use any manual snapshots and care about scheduling.
Key question is if cache size = origin size, could origin be fully restored automatically?
mdadm cache ssd block-device bcache
asked Feb 19 at 9:01
nikolobok
11
edited Feb 19 at 9:22
asked Feb 19 at 9:01
nikolobok
11
asked Feb 19 at 9:01
nikolobok
11
asked Feb 19 at 9:01
nikolobok
11
11
Why not just buy a 2nd SSD and use some form of RAID-1 aka mirroring (with mdadm, lvm, btrfs, or zfs, etc)? Drive mirroring is a tried and tested technology that Just Works. DIY with dm-cache isn't (not for this kind of task, anyway). BTW, with btrfs or zfs you can also perform regular incremental backups with,btrfs sendorzfs sendto a remote machine...and (unlike tar or rsync) they're a low-impact backup that can be performed multiple times per hour without unduly affecting overall system performance.
– cas
Feb 19 at 9:31
Because it is just 2 times more expensive
– nikolobok
Feb 19 at 9:50
Yes. there's a cost to improve the safety of your data. IMO, it's worth every cent. Personally, I'd never store anything important on anything less than a mirrored pair of drives (I use ZFS - gives me snapshots, compression, error correction, and lots of other useful stuff as well as mirroring) in addition to regular backups. If you want to do it cheaper, use 2 x HDDs and then add a smallish fast SSD for caching (e.g. with mdadm+bcache, or ZFS can use SSDs to cache HDD pools).
– cas
Feb 19 at 14:50
I want hdd raid0 + ssd (dm-cache) sensitive data on ssd + backed up by hdd
– nikolobok
Feb 20 at 14:54
if you want raid-0 then you are effectively saying "I do not care at all about my data. It is garbage and if I lost it all I wouldn't care in the slighest".
– cas
Feb 21 at 0:49
 |Â
show 2 more comments
Why not just buy a 2nd SSD and use some form of RAID-1 aka mirroring (with mdadm, lvm, btrfs, or zfs, etc)? Drive mirroring is a tried and tested technology that Just Works. DIY with dm-cache isn't (not for this kind of task, anyway). BTW, with btrfs or zfs you can also perform regular incremental backups with,btrfs sendorzfs sendto a remote machine...and (unlike tar or rsync) they're a low-impact backup that can be performed multiple times per hour without unduly affecting overall system performance.
– cas
Feb 19 at 9:31
Because it is just 2 times more expensive
– nikolobok
Feb 19 at 9:50
Yes. there's a cost to improve the safety of your data. IMO, it's worth every cent. Personally, I'd never store anything important on anything less than a mirrored pair of drives (I use ZFS - gives me snapshots, compression, error correction, and lots of other useful stuff as well as mirroring) in addition to regular backups. If you want to do it cheaper, use 2 x HDDs and then add a smallish fast SSD for caching (e.g. with mdadm+bcache, or ZFS can use SSDs to cache HDD pools).
– cas
Feb 19 at 14:50
I want hdd raid0 + ssd (dm-cache) sensitive data on ssd + backed up by hdd
– nikolobok
Feb 20 at 14:54
if you want raid-0 then you are effectively saying "I do not care at all about my data. It is garbage and if I lost it all I wouldn't care in the slighest".
– cas
Feb 21 at 0:49
Why not just buy a 2nd SSD and use some form of RAID-1 aka mirroring (with mdadm, lvm, btrfs, or zfs, etc)? Drive mirroring is a tried and tested technology that Just Works. DIY with dm-cache isn't (not for this kind of task, anyway). BTW, with btrfs or zfs you can also perform regular incremental backups with,
btrfs send or zfs send to a remote machine...and (unlike tar or rsync) they're a low-impact backup that can be performed multiple times per hour without unduly affecting overall system performance.– cas
Feb 19 at 9:31
Why not just buy a 2nd SSD and use some form of RAID-1 aka mirroring (with mdadm, lvm, btrfs, or zfs, etc)? Drive mirroring is a tried and tested technology that Just Works. DIY with dm-cache isn't (not for this kind of task, anyway). BTW, with btrfs or zfs you can also perform regular incremental backups with,
btrfs send or zfs send to a remote machine...and (unlike tar or rsync) they're a low-impact backup that can be performed multiple times per hour without unduly affecting overall system performance.– cas
Feb 19 at 9:31
Because it is just 2 times more expensive
– nikolobok
Feb 19 at 9:50
Because it is just 2 times more expensive
– nikolobok
Feb 19 at 9:50
Yes. there's a cost to improve the safety of your data. IMO, it's worth every cent. Personally, I'd never store anything important on anything less than a mirrored pair of drives (I use ZFS - gives me snapshots, compression, error correction, and lots of other useful stuff as well as mirroring) in addition to regular backups. If you want to do it cheaper, use 2 x HDDs and then add a smallish fast SSD for caching (e.g. with mdadm+bcache, or ZFS can use SSDs to cache HDD pools).
– cas
Feb 19 at 14:50
Yes. there's a cost to improve the safety of your data. IMO, it's worth every cent. Personally, I'd never store anything important on anything less than a mirrored pair of drives (I use ZFS - gives me snapshots, compression, error correction, and lots of other useful stuff as well as mirroring) in addition to regular backups. If you want to do it cheaper, use 2 x HDDs and then add a smallish fast SSD for caching (e.g. with mdadm+bcache, or ZFS can use SSDs to cache HDD pools).
– cas
Feb 19 at 14:50
I want hdd raid0 + ssd (dm-cache) sensitive data on ssd + backed up by hdd
– nikolobok
Feb 20 at 14:54
I want hdd raid0 + ssd (dm-cache) sensitive data on ssd + backed up by hdd
– nikolobok
Feb 20 at 14:54
if you want raid-0 then you are effectively saying "I do not care at all about my data. It is garbage and if I lost it all I wouldn't care in the slighest".
– cas
Feb 21 at 0:49
if you want raid-0 then you are effectively saying "I do not care at all about my data. It is garbage and if I lost it all I wouldn't care in the slighest".
– cas
Feb 21 at 0:49
 |Â
show 2 more comments
2 Answers
2
active
oldest
votes
up vote
0
down vote
Seems it could be used if I find a way to mark all blocks in cache metadata as dirty and then demote them to new hdd in case of origin hdd failure.
https://www.kernel.org/doc/Documentation/device-mapper/cache.txt
answered Feb 19 at 10:28
nikolobok
11
add a comment |Â
up vote
0
down vote
No, this won't work. There's no method for "pushing" non-dirty blocks in the cache to the origin device. You will have dirty blocks temporarily which have not been written to the origin device, which you can flush to the origin device using the "cleaner" cache option, but there's not way to re-create the origin device with a cache device (even if it is the same size).
answered Apr 11 at 1:35
GoldenNewby
397146
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Seems it could be used if I find a way to mark all blocks in cache metadata as dirty and then demote them to new hdd in case of origin hdd failure.
https://www.kernel.org/doc/Documentation/device-mapper/cache.txt
answered Feb 19 at 10:28
nikolobok
11
add a comment |Â
up vote
0
down vote
Seems it could be used if I find a way to mark all blocks in cache metadata as dirty and then demote them to new hdd in case of origin hdd failure.
https://www.kernel.org/doc/Documentation/device-mapper/cache.txt
answered Feb 19 at 10:28
nikolobok
11
add a comment |Â
up vote
0
down vote
up vote
0
down vote
Seems it could be used if I find a way to mark all blocks in cache metadata as dirty and then demote them to new hdd in case of origin hdd failure.
https://www.kernel.org/doc/Documentation/device-mapper/cache.txt
answered Feb 19 at 10:28
nikolobok
11
Seems it could be used if I find a way to mark all blocks in cache metadata as dirty and then demote them to new hdd in case of origin hdd failure.
https://www.kernel.org/doc/Documentation/device-mapper/cache.txt
answered Feb 19 at 10:28
nikolobok
11
answered Feb 19 at 10:28
nikolobok
11
answered Feb 19 at 10:28
nikolobok
11
answered Feb 19 at 10:28
nikolobok
11
11
add a comment |Â
add a comment |Â
up vote
0
down vote
No, this won't work. There's no method for "pushing" non-dirty blocks in the cache to the origin device. You will have dirty blocks temporarily which have not been written to the origin device, which you can flush to the origin device using the "cleaner" cache option, but there's not way to re-create the origin device with a cache device (even if it is the same size).
answered Apr 11 at 1:35
GoldenNewby
397146
add a comment |Â
up vote
0
down vote
No, this won't work. There's no method for "pushing" non-dirty blocks in the cache to the origin device. You will have dirty blocks temporarily which have not been written to the origin device, which you can flush to the origin device using the "cleaner" cache option, but there's not way to re-create the origin device with a cache device (even if it is the same size).
answered Apr 11 at 1:35
GoldenNewby
397146
add a comment |Â
up vote
0
down vote
up vote
0
down vote
No, this won't work. There's no method for "pushing" non-dirty blocks in the cache to the origin device. You will have dirty blocks temporarily which have not been written to the origin device, which you can flush to the origin device using the "cleaner" cache option, but there's not way to re-create the origin device with a cache device (even if it is the same size).
answered Apr 11 at 1:35
GoldenNewby
397146
No, this won't work. There's no method for "pushing" non-dirty blocks in the cache to the origin device. You will have dirty blocks temporarily which have not been written to the origin device, which you can flush to the origin device using the "cleaner" cache option, but there's not way to re-create the origin device with a cache device (even if it is the same size).
answered Apr 11 at 1:35
GoldenNewby
397146
answered Apr 11 at 1:35
GoldenNewby
397146
answered Apr 11 at 1:35
GoldenNewby
397146
answered Apr 11 at 1:35
GoldenNewby
397146
397146
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f425118%2fusing-dm-cache-for-transparent-ongoing-backup%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Why not just buy a 2nd SSD and use some form of RAID-1 aka mirroring (with mdadm, lvm, btrfs, or zfs, etc)? Drive mirroring is a tried and tested technology that Just Works. DIY with dm-cache isn't (not for this kind of task, anyway). BTW, with btrfs or zfs you can also perform regular incremental backups with,
btrfs sendorzfs sendto a remote machine...and (unlike tar or rsync) they're a low-impact backup that can be performed multiple times per hour without unduly affecting overall system performance.– cas
Feb 19 at 9:31
Because it is just 2 times more expensive
– nikolobok
Feb 19 at 9:50
Yes. there's a cost to improve the safety of your data. IMO, it's worth every cent. Personally, I'd never store anything important on anything less than a mirrored pair of drives (I use ZFS - gives me snapshots, compression, error correction, and lots of other useful stuff as well as mirroring) in addition to regular backups. If you want to do it cheaper, use 2 x HDDs and then add a smallish fast SSD for caching (e.g. with mdadm+bcache, or ZFS can use SSDs to cache HDD pools).
– cas
Feb 19 at 14:50
I want hdd raid0 + ssd (dm-cache) sensitive data on ssd + backed up by hdd
– nikolobok
Feb 20 at 14:54
if you want raid-0 then you are effectively saying "I do not care at all about my data. It is garbage and if I lost it all I wouldn't care in the slighest".
– cas
Feb 21 at 0:49