Zip Archive With Limited Number of Files
Clash Royale CLAN TAG#URR8PPP
up vote
10
down vote
favorite
What command can I use to create zip
s with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip
archives, each consisting of no more than 500 files.
I also don't want the resulting 10 zip
files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.
files zip
add a comment |
up vote
10
down vote
favorite
What command can I use to create zip
s with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip
archives, each consisting of no more than 500 files.
I also don't want the resulting 10 zip
files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.
files zip
add a comment |
up vote
10
down vote
favorite
up vote
10
down vote
favorite
What command can I use to create zip
s with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip
archives, each consisting of no more than 500 files.
I also don't want the resulting 10 zip
files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.
files zip
What command can I use to create zip
s with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip
archives, each consisting of no more than 500 files.
I also don't want the resulting 10 zip
files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.
files zip
files zip
edited Apr 10 '17 at 19:25
don_crissti
49k15129157
49k15129157
asked Nov 9 '14 at 15:42
user8547
60651335
60651335
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
up vote
12
down vote
accepted
You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):
$ touch $(seq 20)
$ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
adding: 1 (stored 0%)
adding: 10 (stored 0%)
adding: 11 (stored 0%)
adding: 12 (stored 0%)
adding: 13 (stored 0%)
adding: 14 (stored 0%)
adding: 15 (stored 0%)
adding: 16 (stored 0%)
adding: 17 (stored 0%)
adding: 18 (stored 0%)
adding: 19 (stored 0%)
adding: 2 (stored 0%)
adding: 20 (stored 0%)
adding: 3 (stored 0%)
adding: 4 (stored 0%)
adding: 5 (stored 0%)
adding: 6 (stored 0%)
adding: 7 (stored 0%)
adding: 8 (stored 0%)
adding: 9 (stored 0%)
$ ls
1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip
The option -N 5
limits the number of files to 5 per archive and is presented to zip
in place of
The #
(verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip
, arch2.zip
etc.
The -print0
option to find
and -0
option to parallel
in tandem make sure that filenames with special characters are correctly handled.
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
2
@user8547 no, just runsudo apt-get install parallel
.
– terdon♦
Nov 9 '14 at 16:20
2
@user8547 whyarch13
? You really need to use the#
character. What shell are you using?
– Anthon
Nov 9 '14 at 16:32
2
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
|
show 6 more comments
up vote
1
down vote
A shell-only alternative: process batches of COUNT files via "$@:START:COUNT"
(range of positional parameters) and shift COUNT
while incrementing a counter c
to name the archives:
set -- *
c=1
while (($#)); do
if [ $# -ge COUNT ]; then
zip $c.zip "$@:1:COUNT"
c=$((c+1))
shift COUNT
else
zip $c.zip "$@"
shift $#
fi
done
add a comment |
up vote
1
down vote
The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:
find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'
Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
You might want to use the -0 trick Anthon suggested, if you have weird filenames.
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
12
down vote
accepted
You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):
$ touch $(seq 20)
$ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
adding: 1 (stored 0%)
adding: 10 (stored 0%)
adding: 11 (stored 0%)
adding: 12 (stored 0%)
adding: 13 (stored 0%)
adding: 14 (stored 0%)
adding: 15 (stored 0%)
adding: 16 (stored 0%)
adding: 17 (stored 0%)
adding: 18 (stored 0%)
adding: 19 (stored 0%)
adding: 2 (stored 0%)
adding: 20 (stored 0%)
adding: 3 (stored 0%)
adding: 4 (stored 0%)
adding: 5 (stored 0%)
adding: 6 (stored 0%)
adding: 7 (stored 0%)
adding: 8 (stored 0%)
adding: 9 (stored 0%)
$ ls
1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip
The option -N 5
limits the number of files to 5 per archive and is presented to zip
in place of
The #
(verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip
, arch2.zip
etc.
The -print0
option to find
and -0
option to parallel
in tandem make sure that filenames with special characters are correctly handled.
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
2
@user8547 no, just runsudo apt-get install parallel
.
– terdon♦
Nov 9 '14 at 16:20
2
@user8547 whyarch13
? You really need to use the#
character. What shell are you using?
– Anthon
Nov 9 '14 at 16:32
2
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
|
show 6 more comments
up vote
12
down vote
accepted
You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):
$ touch $(seq 20)
$ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
adding: 1 (stored 0%)
adding: 10 (stored 0%)
adding: 11 (stored 0%)
adding: 12 (stored 0%)
adding: 13 (stored 0%)
adding: 14 (stored 0%)
adding: 15 (stored 0%)
adding: 16 (stored 0%)
adding: 17 (stored 0%)
adding: 18 (stored 0%)
adding: 19 (stored 0%)
adding: 2 (stored 0%)
adding: 20 (stored 0%)
adding: 3 (stored 0%)
adding: 4 (stored 0%)
adding: 5 (stored 0%)
adding: 6 (stored 0%)
adding: 7 (stored 0%)
adding: 8 (stored 0%)
adding: 9 (stored 0%)
$ ls
1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip
The option -N 5
limits the number of files to 5 per archive and is presented to zip
in place of
The #
(verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip
, arch2.zip
etc.
The -print0
option to find
and -0
option to parallel
in tandem make sure that filenames with special characters are correctly handled.
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
2
@user8547 no, just runsudo apt-get install parallel
.
– terdon♦
Nov 9 '14 at 16:20
2
@user8547 whyarch13
? You really need to use the#
character. What shell are you using?
– Anthon
Nov 9 '14 at 16:32
2
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
|
show 6 more comments
up vote
12
down vote
accepted
up vote
12
down vote
accepted
You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):
$ touch $(seq 20)
$ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
adding: 1 (stored 0%)
adding: 10 (stored 0%)
adding: 11 (stored 0%)
adding: 12 (stored 0%)
adding: 13 (stored 0%)
adding: 14 (stored 0%)
adding: 15 (stored 0%)
adding: 16 (stored 0%)
adding: 17 (stored 0%)
adding: 18 (stored 0%)
adding: 19 (stored 0%)
adding: 2 (stored 0%)
adding: 20 (stored 0%)
adding: 3 (stored 0%)
adding: 4 (stored 0%)
adding: 5 (stored 0%)
adding: 6 (stored 0%)
adding: 7 (stored 0%)
adding: 8 (stored 0%)
adding: 9 (stored 0%)
$ ls
1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip
The option -N 5
limits the number of files to 5 per archive and is presented to zip
in place of
The #
(verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip
, arch2.zip
etc.
The -print0
option to find
and -0
option to parallel
in tandem make sure that filenames with special characters are correctly handled.
You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):
$ touch $(seq 20)
$ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
adding: 1 (stored 0%)
adding: 10 (stored 0%)
adding: 11 (stored 0%)
adding: 12 (stored 0%)
adding: 13 (stored 0%)
adding: 14 (stored 0%)
adding: 15 (stored 0%)
adding: 16 (stored 0%)
adding: 17 (stored 0%)
adding: 18 (stored 0%)
adding: 19 (stored 0%)
adding: 2 (stored 0%)
adding: 20 (stored 0%)
adding: 3 (stored 0%)
adding: 4 (stored 0%)
adding: 5 (stored 0%)
adding: 6 (stored 0%)
adding: 7 (stored 0%)
adding: 8 (stored 0%)
adding: 9 (stored 0%)
$ ls
1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip
The option -N 5
limits the number of files to 5 per archive and is presented to zip
in place of
The #
(verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip
, arch2.zip
etc.
The -print0
option to find
and -0
option to parallel
in tandem make sure that filenames with special characters are correctly handled.
edited Nov 9 '14 at 16:44
answered Nov 9 '14 at 15:55
Anthon
59.9k17102163
59.9k17102163
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
2
@user8547 no, just runsudo apt-get install parallel
.
– terdon♦
Nov 9 '14 at 16:20
2
@user8547 whyarch13
? You really need to use the#
character. What shell are you using?
– Anthon
Nov 9 '14 at 16:32
2
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
|
show 6 more comments
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
2
@user8547 no, just runsudo apt-get install parallel
.
– terdon♦
Nov 9 '14 at 16:20
2
@user8547 whyarch13
? You really need to use the#
character. What shell are you using?
– Anthon
Nov 9 '14 at 16:32
2
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
– user8547
Nov 9 '14 at 16:07
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
@user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
– Anthon
Nov 9 '14 at 16:15
2
2
@user8547 no, just run
sudo apt-get install parallel
.– terdon♦
Nov 9 '14 at 16:20
@user8547 no, just run
sudo apt-get install parallel
.– terdon♦
Nov 9 '14 at 16:20
2
2
@user8547 why
arch13
? You really need to use the #
character. What shell are you using?– Anthon
Nov 9 '14 at 16:32
@user8547 why
arch13
? You really need to use the #
character. What shell are you using?– Anthon
Nov 9 '14 at 16:32
2
2
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
@user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
– Anthon
Nov 9 '14 at 16:37
|
show 6 more comments
up vote
1
down vote
A shell-only alternative: process batches of COUNT files via "$@:START:COUNT"
(range of positional parameters) and shift COUNT
while incrementing a counter c
to name the archives:
set -- *
c=1
while (($#)); do
if [ $# -ge COUNT ]; then
zip $c.zip "$@:1:COUNT"
c=$((c+1))
shift COUNT
else
zip $c.zip "$@"
shift $#
fi
done
add a comment |
up vote
1
down vote
A shell-only alternative: process batches of COUNT files via "$@:START:COUNT"
(range of positional parameters) and shift COUNT
while incrementing a counter c
to name the archives:
set -- *
c=1
while (($#)); do
if [ $# -ge COUNT ]; then
zip $c.zip "$@:1:COUNT"
c=$((c+1))
shift COUNT
else
zip $c.zip "$@"
shift $#
fi
done
add a comment |
up vote
1
down vote
up vote
1
down vote
A shell-only alternative: process batches of COUNT files via "$@:START:COUNT"
(range of positional parameters) and shift COUNT
while incrementing a counter c
to name the archives:
set -- *
c=1
while (($#)); do
if [ $# -ge COUNT ]; then
zip $c.zip "$@:1:COUNT"
c=$((c+1))
shift COUNT
else
zip $c.zip "$@"
shift $#
fi
done
A shell-only alternative: process batches of COUNT files via "$@:START:COUNT"
(range of positional parameters) and shift COUNT
while incrementing a counter c
to name the archives:
set -- *
c=1
while (($#)); do
if [ $# -ge COUNT ]; then
zip $c.zip "$@:1:COUNT"
c=$((c+1))
shift COUNT
else
zip $c.zip "$@"
shift $#
fi
done
answered Apr 26 '16 at 14:22
community wiki
don_crissti
add a comment |
add a comment |
up vote
1
down vote
The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:
find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'
Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
You might want to use the -0 trick Anthon suggested, if you have weird filenames.
add a comment |
up vote
1
down vote
The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:
find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'
Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
You might want to use the -0 trick Anthon suggested, if you have weird filenames.
add a comment |
up vote
1
down vote
up vote
1
down vote
The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:
find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'
Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
You might want to use the -0 trick Anthon suggested, if you have weird filenames.
The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:
find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'
Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
You might want to use the -0 trick Anthon suggested, if you have weird filenames.
answered Mar 18 '17 at 1:56
msb
1,16079
1,16079
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f166976%2fzip-archive-with-limited-number-of-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown