How to delete old backups based on a date in file name?
Clash Royale CLAN TAG#URR8PPP
up vote
5
down vote
favorite
I have a daily backups named like this:
yyyymmddhhmm.zip // pattern
201503200100.zip // backup from 20. 3. 2015 1:00
I'm trying to create a script that deletes all backups older than 3 days. The script should be also able to delete all other files in the folder not matching the pattern (but there would be a switch for that in the script to disable this).
To determine the file age I don't want to use backups timestamps as other programs also manipulate with the files and it can be tampered.
With the help of: Remove files older than 5 days in UNIX (date in file name, not timestamp)
I got:
#!/bin/bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups/
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
ls -1 $BACKUPS_PATH????????????.zip |
while read A DATE B FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
if [ $DELETE_OTHERS == "yes" ]; then
rm $BACKUPS_PATH*.* // but I don't know how to not-delete the files matching pattern
fi
But it keeps saying:
rm: missing operand
Where is the problem and how to complete the script?
bash shell-script scripting file-management
add a comment |
up vote
5
down vote
favorite
I have a daily backups named like this:
yyyymmddhhmm.zip // pattern
201503200100.zip // backup from 20. 3. 2015 1:00
I'm trying to create a script that deletes all backups older than 3 days. The script should be also able to delete all other files in the folder not matching the pattern (but there would be a switch for that in the script to disable this).
To determine the file age I don't want to use backups timestamps as other programs also manipulate with the files and it can be tampered.
With the help of: Remove files older than 5 days in UNIX (date in file name, not timestamp)
I got:
#!/bin/bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups/
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
ls -1 $BACKUPS_PATH????????????.zip |
while read A DATE B FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
if [ $DELETE_OTHERS == "yes" ]; then
rm $BACKUPS_PATH*.* // but I don't know how to not-delete the files matching pattern
fi
But it keeps saying:
rm: missing operand
Where is the problem and how to complete the script?
bash shell-script scripting file-management
Do you actually have a!
at the beginning of yourARCHIVE
andbackups
directory names? If so, why in the world would you want to complicate your life in such a way?
– terdon♦
Mar 21 '15 at 17:09
Yes I have. I'm using!
to keep the one or two key directories on top of the file listings when there is a lot of other files/folders in that directory and it is sorted by name. I am all ears if there is a better technique for that :)
– Joudicek Jouda
Mar 21 '15 at 19:13
1
I just addaa
or similar to the name, for exampleaabackups
. That way, you don't need to escape special characters. It's a matter of personal preference though.
– terdon♦
Mar 21 '15 at 20:34
add a comment |
up vote
5
down vote
favorite
up vote
5
down vote
favorite
I have a daily backups named like this:
yyyymmddhhmm.zip // pattern
201503200100.zip // backup from 20. 3. 2015 1:00
I'm trying to create a script that deletes all backups older than 3 days. The script should be also able to delete all other files in the folder not matching the pattern (but there would be a switch for that in the script to disable this).
To determine the file age I don't want to use backups timestamps as other programs also manipulate with the files and it can be tampered.
With the help of: Remove files older than 5 days in UNIX (date in file name, not timestamp)
I got:
#!/bin/bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups/
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
ls -1 $BACKUPS_PATH????????????.zip |
while read A DATE B FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
if [ $DELETE_OTHERS == "yes" ]; then
rm $BACKUPS_PATH*.* // but I don't know how to not-delete the files matching pattern
fi
But it keeps saying:
rm: missing operand
Where is the problem and how to complete the script?
bash shell-script scripting file-management
I have a daily backups named like this:
yyyymmddhhmm.zip // pattern
201503200100.zip // backup from 20. 3. 2015 1:00
I'm trying to create a script that deletes all backups older than 3 days. The script should be also able to delete all other files in the folder not matching the pattern (but there would be a switch for that in the script to disable this).
To determine the file age I don't want to use backups timestamps as other programs also manipulate with the files and it can be tampered.
With the help of: Remove files older than 5 days in UNIX (date in file name, not timestamp)
I got:
#!/bin/bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups/
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
ls -1 $BACKUPS_PATH????????????.zip |
while read A DATE B FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
if [ $DELETE_OTHERS == "yes" ]; then
rm $BACKUPS_PATH*.* // but I don't know how to not-delete the files matching pattern
fi
But it keeps saying:
rm: missing operand
Where is the problem and how to complete the script?
bash shell-script scripting file-management
bash shell-script scripting file-management
edited Apr 13 '17 at 12:36
Community♦
1
1
asked Mar 21 '15 at 15:42
Joudicek Jouda
166126
166126
Do you actually have a!
at the beginning of yourARCHIVE
andbackups
directory names? If so, why in the world would you want to complicate your life in such a way?
– terdon♦
Mar 21 '15 at 17:09
Yes I have. I'm using!
to keep the one or two key directories on top of the file listings when there is a lot of other files/folders in that directory and it is sorted by name. I am all ears if there is a better technique for that :)
– Joudicek Jouda
Mar 21 '15 at 19:13
1
I just addaa
or similar to the name, for exampleaabackups
. That way, you don't need to escape special characters. It's a matter of personal preference though.
– terdon♦
Mar 21 '15 at 20:34
add a comment |
Do you actually have a!
at the beginning of yourARCHIVE
andbackups
directory names? If so, why in the world would you want to complicate your life in such a way?
– terdon♦
Mar 21 '15 at 17:09
Yes I have. I'm using!
to keep the one or two key directories on top of the file listings when there is a lot of other files/folders in that directory and it is sorted by name. I am all ears if there is a better technique for that :)
– Joudicek Jouda
Mar 21 '15 at 19:13
1
I just addaa
or similar to the name, for exampleaabackups
. That way, you don't need to escape special characters. It's a matter of personal preference though.
– terdon♦
Mar 21 '15 at 20:34
Do you actually have a
!
at the beginning of your ARCHIVE
and backups
directory names? If so, why in the world would you want to complicate your life in such a way?– terdon♦
Mar 21 '15 at 17:09
Do you actually have a
!
at the beginning of your ARCHIVE
and backups
directory names? If so, why in the world would you want to complicate your life in such a way?– terdon♦
Mar 21 '15 at 17:09
Yes I have. I'm using
!
to keep the one or two key directories on top of the file listings when there is a lot of other files/folders in that directory and it is sorted by name. I am all ears if there is a better technique for that :)– Joudicek Jouda
Mar 21 '15 at 19:13
Yes I have. I'm using
!
to keep the one or two key directories on top of the file listings when there is a lot of other files/folders in that directory and it is sorted by name. I am all ears if there is a better technique for that :)– Joudicek Jouda
Mar 21 '15 at 19:13
1
1
I just add
aa
or similar to the name, for example aabackups
. That way, you don't need to escape special characters. It's a matter of personal preference though.– terdon♦
Mar 21 '15 at 20:34
I just add
aa
or similar to the name, for example aabackups
. That way, you don't need to escape special characters. It's a matter of personal preference though.– terdon♦
Mar 21 '15 at 20:34
add a comment |
3 Answers
3
active
oldest
votes
up vote
6
down vote
accepted
The first problem in your code is that you are parsing ls
. This means it will break very easily, if you have any spaces in your file or directory names for example. You should use shell globbing or find
instead.
A bigger problem is that you are not reading the data correctly. Your code:
ls -1 | while read A DATE B FILE
will never populate $FILE
. The output of ls -1
is just a list of filenames so, unless those file names contain whitespace, only the first of the 4 variables you give to read
will be populated.
Here's a working version of your script:
#!/usr/bin/env bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
## Find all files in $BACKUPS_PATH. The -type f means only files
## and the -maxdepth 1 ensures that any files in subdirectories are
## not included. Combined with -print0 (separate file names with ),
## IFS= (don't break on whitespace), "-d ''" (records end on '') , it can
## deal with all file names.
find $BACKUPS_PATH -maxdepth 1 -type f -print0 | while IFS= read -d '' -r file
do
## Does this file name match the pattern (13 digits, then .zip)?
if [[ "$(basename "$file")" =~ ^[0-9]12.zip$ ]]
then
## Delete the file if it's older than the $THR
[ "$(basename "$file" .zip)" -le "$THRESHOLD" ] && rm -v -- "$file"
else
## If the file does not match the pattern, delete if
## DELETE_OTHERS is set to "yes"
[ $DELETE_OTHERS == "yes" ] && rm -v -- "$file"
fi
done
add a comment |
up vote
0
down vote
don't you forget the sed
line between ls -1
and while read
, this line IS important.
for first question I would suggest: (a awk remplacement I was unable to find a sed equivalent)
ls -1 $BACKUPS_PATH????????????.zip |
awk -F. 'printf "%s %sn",$1,$0 ;' |
while read DATE FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
provided shell arithmetic are at least 37 bit wise to do the $DATE -le $THRESHOLD
testing.
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
add a comment |
up vote
0
down vote
In FreeBSD use:
Example: find all files in /usr/home/foobar owned by foobar that are older than 5760 minutes (4 days) and delete them.
find /usr/home/foobar -user foobar -type f -mmin +5760 -delete
New contributor
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
6
down vote
accepted
The first problem in your code is that you are parsing ls
. This means it will break very easily, if you have any spaces in your file or directory names for example. You should use shell globbing or find
instead.
A bigger problem is that you are not reading the data correctly. Your code:
ls -1 | while read A DATE B FILE
will never populate $FILE
. The output of ls -1
is just a list of filenames so, unless those file names contain whitespace, only the first of the 4 variables you give to read
will be populated.
Here's a working version of your script:
#!/usr/bin/env bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
## Find all files in $BACKUPS_PATH. The -type f means only files
## and the -maxdepth 1 ensures that any files in subdirectories are
## not included. Combined with -print0 (separate file names with ),
## IFS= (don't break on whitespace), "-d ''" (records end on '') , it can
## deal with all file names.
find $BACKUPS_PATH -maxdepth 1 -type f -print0 | while IFS= read -d '' -r file
do
## Does this file name match the pattern (13 digits, then .zip)?
if [[ "$(basename "$file")" =~ ^[0-9]12.zip$ ]]
then
## Delete the file if it's older than the $THR
[ "$(basename "$file" .zip)" -le "$THRESHOLD" ] && rm -v -- "$file"
else
## If the file does not match the pattern, delete if
## DELETE_OTHERS is set to "yes"
[ $DELETE_OTHERS == "yes" ] && rm -v -- "$file"
fi
done
add a comment |
up vote
6
down vote
accepted
The first problem in your code is that you are parsing ls
. This means it will break very easily, if you have any spaces in your file or directory names for example. You should use shell globbing or find
instead.
A bigger problem is that you are not reading the data correctly. Your code:
ls -1 | while read A DATE B FILE
will never populate $FILE
. The output of ls -1
is just a list of filenames so, unless those file names contain whitespace, only the first of the 4 variables you give to read
will be populated.
Here's a working version of your script:
#!/usr/bin/env bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
## Find all files in $BACKUPS_PATH. The -type f means only files
## and the -maxdepth 1 ensures that any files in subdirectories are
## not included. Combined with -print0 (separate file names with ),
## IFS= (don't break on whitespace), "-d ''" (records end on '') , it can
## deal with all file names.
find $BACKUPS_PATH -maxdepth 1 -type f -print0 | while IFS= read -d '' -r file
do
## Does this file name match the pattern (13 digits, then .zip)?
if [[ "$(basename "$file")" =~ ^[0-9]12.zip$ ]]
then
## Delete the file if it's older than the $THR
[ "$(basename "$file" .zip)" -le "$THRESHOLD" ] && rm -v -- "$file"
else
## If the file does not match the pattern, delete if
## DELETE_OTHERS is set to "yes"
[ $DELETE_OTHERS == "yes" ] && rm -v -- "$file"
fi
done
add a comment |
up vote
6
down vote
accepted
up vote
6
down vote
accepted
The first problem in your code is that you are parsing ls
. This means it will break very easily, if you have any spaces in your file or directory names for example. You should use shell globbing or find
instead.
A bigger problem is that you are not reading the data correctly. Your code:
ls -1 | while read A DATE B FILE
will never populate $FILE
. The output of ls -1
is just a list of filenames so, unless those file names contain whitespace, only the first of the 4 variables you give to read
will be populated.
Here's a working version of your script:
#!/usr/bin/env bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
## Find all files in $BACKUPS_PATH. The -type f means only files
## and the -maxdepth 1 ensures that any files in subdirectories are
## not included. Combined with -print0 (separate file names with ),
## IFS= (don't break on whitespace), "-d ''" (records end on '') , it can
## deal with all file names.
find $BACKUPS_PATH -maxdepth 1 -type f -print0 | while IFS= read -d '' -r file
do
## Does this file name match the pattern (13 digits, then .zip)?
if [[ "$(basename "$file")" =~ ^[0-9]12.zip$ ]]
then
## Delete the file if it's older than the $THR
[ "$(basename "$file" .zip)" -le "$THRESHOLD" ] && rm -v -- "$file"
else
## If the file does not match the pattern, delete if
## DELETE_OTHERS is set to "yes"
[ $DELETE_OTHERS == "yes" ] && rm -v -- "$file"
fi
done
The first problem in your code is that you are parsing ls
. This means it will break very easily, if you have any spaces in your file or directory names for example. You should use shell globbing or find
instead.
A bigger problem is that you are not reading the data correctly. Your code:
ls -1 | while read A DATE B FILE
will never populate $FILE
. The output of ls -1
is just a list of filenames so, unless those file names contain whitespace, only the first of the 4 variables you give to read
will be populated.
Here's a working version of your script:
#!/usr/bin/env bash
DELETE_OTHERS=yes
BACKUPS_PATH=/mnt/!ARCHIVE/!backups
THRESHOLD=$(date -d "3 days ago" +%Y%m%d%H%M)
## Find all files in $BACKUPS_PATH. The -type f means only files
## and the -maxdepth 1 ensures that any files in subdirectories are
## not included. Combined with -print0 (separate file names with ),
## IFS= (don't break on whitespace), "-d ''" (records end on '') , it can
## deal with all file names.
find $BACKUPS_PATH -maxdepth 1 -type f -print0 | while IFS= read -d '' -r file
do
## Does this file name match the pattern (13 digits, then .zip)?
if [[ "$(basename "$file")" =~ ^[0-9]12.zip$ ]]
then
## Delete the file if it's older than the $THR
[ "$(basename "$file" .zip)" -le "$THRESHOLD" ] && rm -v -- "$file"
else
## If the file does not match the pattern, delete if
## DELETE_OTHERS is set to "yes"
[ $DELETE_OTHERS == "yes" ] && rm -v -- "$file"
fi
done
edited Mar 21 '15 at 17:47
answered Mar 21 '15 at 17:38
terdon♦
126k31243418
126k31243418
add a comment |
add a comment |
up vote
0
down vote
don't you forget the sed
line between ls -1
and while read
, this line IS important.
for first question I would suggest: (a awk remplacement I was unable to find a sed equivalent)
ls -1 $BACKUPS_PATH????????????.zip |
awk -F. 'printf "%s %sn",$1,$0 ;' |
while read DATE FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
provided shell arithmetic are at least 37 bit wise to do the $DATE -le $THRESHOLD
testing.
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
add a comment |
up vote
0
down vote
don't you forget the sed
line between ls -1
and while read
, this line IS important.
for first question I would suggest: (a awk remplacement I was unable to find a sed equivalent)
ls -1 $BACKUPS_PATH????????????.zip |
awk -F. 'printf "%s %sn",$1,$0 ;' |
while read DATE FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
provided shell arithmetic are at least 37 bit wise to do the $DATE -le $THRESHOLD
testing.
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
add a comment |
up vote
0
down vote
up vote
0
down vote
don't you forget the sed
line between ls -1
and while read
, this line IS important.
for first question I would suggest: (a awk remplacement I was unable to find a sed equivalent)
ls -1 $BACKUPS_PATH????????????.zip |
awk -F. 'printf "%s %sn",$1,$0 ;' |
while read DATE FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
provided shell arithmetic are at least 37 bit wise to do the $DATE -le $THRESHOLD
testing.
don't you forget the sed
line between ls -1
and while read
, this line IS important.
for first question I would suggest: (a awk remplacement I was unable to find a sed equivalent)
ls -1 $BACKUPS_PATH????????????.zip |
awk -F. 'printf "%s %sn",$1,$0 ;' |
while read DATE FILE
do
[[ $DATE -le $THRESHOLD ]] && rm -v $BACKUPS_PATH$FILE
done
provided shell arithmetic are at least 37 bit wise to do the $DATE -le $THRESHOLD
testing.
edited Mar 21 '15 at 16:57
answered Mar 21 '15 at 16:50
Archemar
19.4k93468
19.4k93468
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
add a comment |
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
This breaks on whitespace and other strange characters.
– terdon♦
Mar 21 '15 at 17:45
add a comment |
up vote
0
down vote
In FreeBSD use:
Example: find all files in /usr/home/foobar owned by foobar that are older than 5760 minutes (4 days) and delete them.
find /usr/home/foobar -user foobar -type f -mmin +5760 -delete
New contributor
add a comment |
up vote
0
down vote
In FreeBSD use:
Example: find all files in /usr/home/foobar owned by foobar that are older than 5760 minutes (4 days) and delete them.
find /usr/home/foobar -user foobar -type f -mmin +5760 -delete
New contributor
add a comment |
up vote
0
down vote
up vote
0
down vote
In FreeBSD use:
Example: find all files in /usr/home/foobar owned by foobar that are older than 5760 minutes (4 days) and delete them.
find /usr/home/foobar -user foobar -type f -mmin +5760 -delete
New contributor
In FreeBSD use:
Example: find all files in /usr/home/foobar owned by foobar that are older than 5760 minutes (4 days) and delete them.
find /usr/home/foobar -user foobar -type f -mmin +5760 -delete
New contributor
New contributor
answered Nov 19 at 16:40
Santiago Staviski
1
1
New contributor
New contributor
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f191632%2fhow-to-delete-old-backups-based-on-a-date-in-file-name%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Do you actually have a
!
at the beginning of yourARCHIVE
andbackups
directory names? If so, why in the world would you want to complicate your life in such a way?– terdon♦
Mar 21 '15 at 17:09
Yes I have. I'm using
!
to keep the one or two key directories on top of the file listings when there is a lot of other files/folders in that directory and it is sorted by name. I am all ears if there is a better technique for that :)– Joudicek Jouda
Mar 21 '15 at 19:13
1
I just add
aa
or similar to the name, for exampleaabackups
. That way, you don't need to escape special characters. It's a matter of personal preference though.– terdon♦
Mar 21 '15 at 20:34