Zip Archive With Limited Number of Files

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
10
down vote

favorite
2












What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip archives, each consisting of no more than 500 files.



I also don't want the resulting 10 zip files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.










share|improve this question



























    up vote
    10
    down vote

    favorite
    2












    What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip archives, each consisting of no more than 500 files.



    I also don't want the resulting 10 zip files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.










    share|improve this question

























      up vote
      10
      down vote

      favorite
      2









      up vote
      10
      down vote

      favorite
      2






      2





      What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip archives, each consisting of no more than 500 files.



      I also don't want the resulting 10 zip files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.










      share|improve this question















      What command can I use to create zips with a file number limit? I have a folder (no subfolders) of, say, 5000 files, so I would want a command that could divide that number and create 10 individual zip archives, each consisting of no more than 500 files.



      I also don't want the resulting 10 zip files to be connected with each other, so that I can open them individually and won't need to open all 10 at the same time.







      files zip






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Apr 10 '17 at 19:25









      don_crissti

      49k15129157




      49k15129157










      asked Nov 9 '14 at 15:42









      user8547

      60651335




      60651335




















          3 Answers
          3






          active

          oldest

          votes

















          up vote
          12
          down vote



          accepted










          You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):



          $ touch $(seq 20)
          $ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
          adding: 1 (stored 0%)
          adding: 10 (stored 0%)
          adding: 11 (stored 0%)
          adding: 12 (stored 0%)
          adding: 13 (stored 0%)
          adding: 14 (stored 0%)
          adding: 15 (stored 0%)
          adding: 16 (stored 0%)
          adding: 17 (stored 0%)
          adding: 18 (stored 0%)
          adding: 19 (stored 0%)
          adding: 2 (stored 0%)
          adding: 20 (stored 0%)
          adding: 3 (stored 0%)
          adding: 4 (stored 0%)
          adding: 5 (stored 0%)
          adding: 6 (stored 0%)
          adding: 7 (stored 0%)
          adding: 8 (stored 0%)
          adding: 9 (stored 0%)
          $ ls
          1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
          10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip


          The option -N 5 limits the number of files to 5 per archive and is presented to zip in place of



          The # (verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip, arch2.zip etc.



          The -print0 option to find and -0 option to parallel in tandem make sure that filenames with special characters are correctly handled.






          share|improve this answer






















          • I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
            – user8547
            Nov 9 '14 at 16:07











          • @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
            – Anthon
            Nov 9 '14 at 16:15







          • 2




            @user8547 no, just run sudo apt-get install parallel.
            – terdon
            Nov 9 '14 at 16:20






          • 2




            @user8547 why arch13? You really need to use the # character. What shell are you using?
            – Anthon
            Nov 9 '14 at 16:32







          • 2




            @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
            – Anthon
            Nov 9 '14 at 16:37

















          up vote
          1
          down vote













          A shell-only alternative: process batches of COUNT files via "$@:START:COUNT" (range of positional parameters) and shift COUNT while incrementing a counter c to name the archives:




          set -- *
          c=1
          while (($#)); do
          if [ $# -ge COUNT ]; then
          zip $c.zip "$@:1:COUNT"
          c=$((c+1))
          shift COUNT
          else
          zip $c.zip "$@"
          shift $#
          fi
          done





          share|improve this answer





























            up vote
            1
            down vote













            The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:



            find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'


            Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
            You might want to use the -0 trick Anthon suggested, if you have weird filenames.






            share|improve this answer




















              Your Answer








              StackExchange.ready(function()
              var channelOptions =
              tags: "".split(" "),
              id: "106"
              ;
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function()
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled)
              StackExchange.using("snippets", function()
              createEditor();
              );

              else
              createEditor();

              );

              function createEditor()
              StackExchange.prepareEditor(
              heartbeatType: 'answer',
              convertImagesToLinks: false,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: null,
              bindNavPrevention: true,
              postfix: "",
              imageUploader:
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              ,
              onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              );



              );













              draft saved

              draft discarded


















              StackExchange.ready(
              function ()
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f166976%2fzip-archive-with-limited-number-of-files%23new-answer', 'question_page');

              );

              Post as a guest















              Required, but never shown

























              3 Answers
              3






              active

              oldest

              votes








              3 Answers
              3






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes








              up vote
              12
              down vote



              accepted










              You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):



              $ touch $(seq 20)
              $ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
              adding: 1 (stored 0%)
              adding: 10 (stored 0%)
              adding: 11 (stored 0%)
              adding: 12 (stored 0%)
              adding: 13 (stored 0%)
              adding: 14 (stored 0%)
              adding: 15 (stored 0%)
              adding: 16 (stored 0%)
              adding: 17 (stored 0%)
              adding: 18 (stored 0%)
              adding: 19 (stored 0%)
              adding: 2 (stored 0%)
              adding: 20 (stored 0%)
              adding: 3 (stored 0%)
              adding: 4 (stored 0%)
              adding: 5 (stored 0%)
              adding: 6 (stored 0%)
              adding: 7 (stored 0%)
              adding: 8 (stored 0%)
              adding: 9 (stored 0%)
              $ ls
              1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
              10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip


              The option -N 5 limits the number of files to 5 per archive and is presented to zip in place of



              The # (verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip, arch2.zip etc.



              The -print0 option to find and -0 option to parallel in tandem make sure that filenames with special characters are correctly handled.






              share|improve this answer






















              • I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
                – user8547
                Nov 9 '14 at 16:07











              • @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
                – Anthon
                Nov 9 '14 at 16:15







              • 2




                @user8547 no, just run sudo apt-get install parallel.
                – terdon
                Nov 9 '14 at 16:20






              • 2




                @user8547 why arch13? You really need to use the # character. What shell are you using?
                – Anthon
                Nov 9 '14 at 16:32







              • 2




                @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
                – Anthon
                Nov 9 '14 at 16:37














              up vote
              12
              down vote



              accepted










              You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):



              $ touch $(seq 20)
              $ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
              adding: 1 (stored 0%)
              adding: 10 (stored 0%)
              adding: 11 (stored 0%)
              adding: 12 (stored 0%)
              adding: 13 (stored 0%)
              adding: 14 (stored 0%)
              adding: 15 (stored 0%)
              adding: 16 (stored 0%)
              adding: 17 (stored 0%)
              adding: 18 (stored 0%)
              adding: 19 (stored 0%)
              adding: 2 (stored 0%)
              adding: 20 (stored 0%)
              adding: 3 (stored 0%)
              adding: 4 (stored 0%)
              adding: 5 (stored 0%)
              adding: 6 (stored 0%)
              adding: 7 (stored 0%)
              adding: 8 (stored 0%)
              adding: 9 (stored 0%)
              $ ls
              1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
              10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip


              The option -N 5 limits the number of files to 5 per archive and is presented to zip in place of



              The # (verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip, arch2.zip etc.



              The -print0 option to find and -0 option to parallel in tandem make sure that filenames with special characters are correctly handled.






              share|improve this answer






















              • I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
                – user8547
                Nov 9 '14 at 16:07











              • @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
                – Anthon
                Nov 9 '14 at 16:15







              • 2




                @user8547 no, just run sudo apt-get install parallel.
                – terdon
                Nov 9 '14 at 16:20






              • 2




                @user8547 why arch13? You really need to use the # character. What shell are you using?
                – Anthon
                Nov 9 '14 at 16:32







              • 2




                @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
                – Anthon
                Nov 9 '14 at 16:37












              up vote
              12
              down vote



              accepted







              up vote
              12
              down vote



              accepted






              You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):



              $ touch $(seq 20)
              $ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
              adding: 1 (stored 0%)
              adding: 10 (stored 0%)
              adding: 11 (stored 0%)
              adding: 12 (stored 0%)
              adding: 13 (stored 0%)
              adding: 14 (stored 0%)
              adding: 15 (stored 0%)
              adding: 16 (stored 0%)
              adding: 17 (stored 0%)
              adding: 18 (stored 0%)
              adding: 19 (stored 0%)
              adding: 2 (stored 0%)
              adding: 20 (stored 0%)
              adding: 3 (stored 0%)
              adding: 4 (stored 0%)
              adding: 5 (stored 0%)
              adding: 6 (stored 0%)
              adding: 7 (stored 0%)
              adding: 8 (stored 0%)
              adding: 9 (stored 0%)
              $ ls
              1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
              10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip


              The option -N 5 limits the number of files to 5 per archive and is presented to zip in place of



              The # (verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip, arch2.zip etc.



              The -print0 option to find and -0 option to parallel in tandem make sure that filenames with special characters are correctly handled.






              share|improve this answer














              You can use GNU parallel to do that as it can limit the number of elements to a job as well as provide a job number (for a unique zip archive name):



              $ touch $(seq 20)
              $ find . ! -name "*.zip" -type f -print0 | parallel -0 -N 5 zip arch#
              adding: 1 (stored 0%)
              adding: 10 (stored 0%)
              adding: 11 (stored 0%)
              adding: 12 (stored 0%)
              adding: 13 (stored 0%)
              adding: 14 (stored 0%)
              adding: 15 (stored 0%)
              adding: 16 (stored 0%)
              adding: 17 (stored 0%)
              adding: 18 (stored 0%)
              adding: 19 (stored 0%)
              adding: 2 (stored 0%)
              adding: 20 (stored 0%)
              adding: 3 (stored 0%)
              adding: 4 (stored 0%)
              adding: 5 (stored 0%)
              adding: 6 (stored 0%)
              adding: 7 (stored 0%)
              adding: 8 (stored 0%)
              adding: 9 (stored 0%)
              $ ls
              1 11 13 15 17 19 20 4 6 8 arch1.zip arch3.zip
              10 12 14 16 18 2 3 5 7 9 arch2.zip arch4.zip


              The option -N 5 limits the number of files to 5 per archive and is presented to zip in place of



              The # (verbatim, not to be replaced by you during the invocation), is replaced by the job number, resulting in arch1.zip, arch2.zip etc.



              The -print0 option to find and -0 option to parallel in tandem make sure that filenames with special characters are correctly handled.







              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Nov 9 '14 at 16:44

























              answered Nov 9 '14 at 15:55









              Anthon

              59.9k17102163




              59.9k17102163











              • I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
                – user8547
                Nov 9 '14 at 16:07











              • @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
                – Anthon
                Nov 9 '14 at 16:15







              • 2




                @user8547 no, just run sudo apt-get install parallel.
                – terdon
                Nov 9 '14 at 16:20






              • 2




                @user8547 why arch13? You really need to use the # character. What shell are you using?
                – Anthon
                Nov 9 '14 at 16:32







              • 2




                @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
                – Anthon
                Nov 9 '14 at 16:37
















              • I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
                – user8547
                Nov 9 '14 at 16:07











              • @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
                – Anthon
                Nov 9 '14 at 16:15







              • 2




                @user8547 no, just run sudo apt-get install parallel.
                – terdon
                Nov 9 '14 at 16:20






              • 2




                @user8547 why arch13? You really need to use the # character. What shell are you using?
                – Anthon
                Nov 9 '14 at 16:32







              • 2




                @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
                – Anthon
                Nov 9 '14 at 16:37















              I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
              – user8547
              Nov 9 '14 at 16:07





              I got this error: i.imgur.com/JoyPrfY.png From this command: find * ! -name "*.zip" -type f -print0 | parallel -0 -N 500 zip arch13
              – user8547
              Nov 9 '14 at 16:07













              @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
              – Anthon
              Nov 9 '14 at 16:15





              @user8547 that is not GNU parallel, but the parallel included in moreutils, you best compile and install from source to get the latest security patches. ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
              – Anthon
              Nov 9 '14 at 16:15





              2




              2




              @user8547 no, just run sudo apt-get install parallel.
              – terdon
              Nov 9 '14 at 16:20




              @user8547 no, just run sudo apt-get install parallel.
              – terdon
              Nov 9 '14 at 16:20




              2




              2




              @user8547 why arch13? You really need to use the # character. What shell are you using?
              – Anthon
              Nov 9 '14 at 16:32





              @user8547 why arch13? You really need to use the # character. What shell are you using?
              – Anthon
              Nov 9 '14 at 16:32





              2




              2




              @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
              – Anthon
              Nov 9 '14 at 16:37




              @user8547 No that is the way to tell parallel to put the job number there, glad it worked out.
              – Anthon
              Nov 9 '14 at 16:37












              up vote
              1
              down vote













              A shell-only alternative: process batches of COUNT files via "$@:START:COUNT" (range of positional parameters) and shift COUNT while incrementing a counter c to name the archives:




              set -- *
              c=1
              while (($#)); do
              if [ $# -ge COUNT ]; then
              zip $c.zip "$@:1:COUNT"
              c=$((c+1))
              shift COUNT
              else
              zip $c.zip "$@"
              shift $#
              fi
              done





              share|improve this answer


























                up vote
                1
                down vote













                A shell-only alternative: process batches of COUNT files via "$@:START:COUNT" (range of positional parameters) and shift COUNT while incrementing a counter c to name the archives:




                set -- *
                c=1
                while (($#)); do
                if [ $# -ge COUNT ]; then
                zip $c.zip "$@:1:COUNT"
                c=$((c+1))
                shift COUNT
                else
                zip $c.zip "$@"
                shift $#
                fi
                done





                share|improve this answer
























                  up vote
                  1
                  down vote










                  up vote
                  1
                  down vote









                  A shell-only alternative: process batches of COUNT files via "$@:START:COUNT" (range of positional parameters) and shift COUNT while incrementing a counter c to name the archives:




                  set -- *
                  c=1
                  while (($#)); do
                  if [ $# -ge COUNT ]; then
                  zip $c.zip "$@:1:COUNT"
                  c=$((c+1))
                  shift COUNT
                  else
                  zip $c.zip "$@"
                  shift $#
                  fi
                  done





                  share|improve this answer














                  A shell-only alternative: process batches of COUNT files via "$@:START:COUNT" (range of positional parameters) and shift COUNT while incrementing a counter c to name the archives:




                  set -- *
                  c=1
                  while (($#)); do
                  if [ $# -ge COUNT ]; then
                  zip $c.zip "$@:1:COUNT"
                  c=$((c+1))
                  shift COUNT
                  else
                  zip $c.zip "$@"
                  shift $#
                  fi
                  done






                  share|improve this answer














                  share|improve this answer



                  share|improve this answer








                  answered Apr 26 '16 at 14:22


























                  community wiki





                  don_crissti





















                      up vote
                      1
                      down vote













                      The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:



                      find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'


                      Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
                      You might want to use the -0 trick Anthon suggested, if you have weird filenames.






                      share|improve this answer
























                        up vote
                        1
                        down vote













                        The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:



                        find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'


                        Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
                        You might want to use the -0 trick Anthon suggested, if you have weird filenames.






                        share|improve this answer






















                          up vote
                          1
                          down vote










                          up vote
                          1
                          down vote









                          The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:



                          find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'


                          Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
                          You might want to use the -0 trick Anthon suggested, if you have weird filenames.






                          share|improve this answer












                          The accepted answer worked perfectly fine for me. :) BUT, in case you don't have access to parallel (who knows why), here's an alternative I had come up with before:



                          find . ! -name '*.zip' -type f | xargs -n 500 | awk 'system("zip myarch"NR".zip "$0)'


                          Which will create myarch1.zip, myarch2.zip, myarch3.zip, etc
                          You might want to use the -0 trick Anthon suggested, if you have weird filenames.







                          share|improve this answer












                          share|improve this answer



                          share|improve this answer










                          answered Mar 18 '17 at 1:56









                          msb

                          1,16079




                          1,16079



























                              draft saved

                              draft discarded
















































                              Thanks for contributing an answer to Unix & Linux Stack Exchange!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid


                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.

                              To learn more, see our tips on writing great answers.





                              Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                              Please pay close attention to the following guidance:


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid


                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.

                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function ()
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f166976%2fzip-archive-with-limited-number-of-files%23new-answer', 'question_page');

                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown






                              Popular posts from this blog

                              How to check contact read email or not when send email to Individual?

                              Bahrain

                              Postfix configuration issue with fips on centos 7; mailgun relay