Pass a list of directories to rsync

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I have a text file list_of_dirs_to_copy with 120 paths:



data/A/backups/2018_02_12
data/B/backups/2/files
data/B/old_files/2015_12
data/C/my_datafiles/800/dat
data/F/backups & old files (second backups)
...


IMPORTANT NOTE: line breaks are LF in the file above and there are special characters such as & and ( in directory names.



data is located in /mnt/storage3/workfiles/backups. So I do cd /mnt/storage3/workfiles/backups so that paths in my list_of_dirs_to_copy become relevant.



Now I want to copy all of the directories listed in my text file (with all of their hidden files and sub-directories) to another hard drive using the following command.



rsync -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats $list_of_directories /drives/f/data_backups/


How can I pass the list of directories to rsync without breaking it (i.e. receiving Argument list too long)? Would it be possible to send each line of the text to rsync one by one? Would that make sense?







share|improve this question

























    up vote
    1
    down vote

    favorite












    I have a text file list_of_dirs_to_copy with 120 paths:



    data/A/backups/2018_02_12
    data/B/backups/2/files
    data/B/old_files/2015_12
    data/C/my_datafiles/800/dat
    data/F/backups & old files (second backups)
    ...


    IMPORTANT NOTE: line breaks are LF in the file above and there are special characters such as & and ( in directory names.



    data is located in /mnt/storage3/workfiles/backups. So I do cd /mnt/storage3/workfiles/backups so that paths in my list_of_dirs_to_copy become relevant.



    Now I want to copy all of the directories listed in my text file (with all of their hidden files and sub-directories) to another hard drive using the following command.



    rsync -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats $list_of_directories /drives/f/data_backups/


    How can I pass the list of directories to rsync without breaking it (i.e. receiving Argument list too long)? Would it be possible to send each line of the text to rsync one by one? Would that make sense?







    share|improve this question























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      I have a text file list_of_dirs_to_copy with 120 paths:



      data/A/backups/2018_02_12
      data/B/backups/2/files
      data/B/old_files/2015_12
      data/C/my_datafiles/800/dat
      data/F/backups & old files (second backups)
      ...


      IMPORTANT NOTE: line breaks are LF in the file above and there are special characters such as & and ( in directory names.



      data is located in /mnt/storage3/workfiles/backups. So I do cd /mnt/storage3/workfiles/backups so that paths in my list_of_dirs_to_copy become relevant.



      Now I want to copy all of the directories listed in my text file (with all of their hidden files and sub-directories) to another hard drive using the following command.



      rsync -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats $list_of_directories /drives/f/data_backups/


      How can I pass the list of directories to rsync without breaking it (i.e. receiving Argument list too long)? Would it be possible to send each line of the text to rsync one by one? Would that make sense?







      share|improve this question













      I have a text file list_of_dirs_to_copy with 120 paths:



      data/A/backups/2018_02_12
      data/B/backups/2/files
      data/B/old_files/2015_12
      data/C/my_datafiles/800/dat
      data/F/backups & old files (second backups)
      ...


      IMPORTANT NOTE: line breaks are LF in the file above and there are special characters such as & and ( in directory names.



      data is located in /mnt/storage3/workfiles/backups. So I do cd /mnt/storage3/workfiles/backups so that paths in my list_of_dirs_to_copy become relevant.



      Now I want to copy all of the directories listed in my text file (with all of their hidden files and sub-directories) to another hard drive using the following command.



      rsync -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats $list_of_directories /drives/f/data_backups/


      How can I pass the list of directories to rsync without breaking it (i.e. receiving Argument list too long)? Would it be possible to send each line of the text to rsync one by one? Would that make sense?









      share|improve this question












      share|improve this question




      share|improve this question








      edited Jul 11 at 12:31
























      asked Jul 11 at 12:28









      benett

      354




      354




















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          2
          down vote













          rsync -av ...options... --files-from=list_of_dirs_to_copy 
          /mnt/storage3/workfiles/backups/
          /drives/f/data_backups


          The paths in list_of_dirs_to_copy would be interpreted as being relative to the source directory. With the given list, this would create data as a subdirectory under /drives/f/data_backups.






          share|improve this answer





















          • This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
            – benett
            Jul 11 at 13:00










          • The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
            – benett
            Jul 11 at 13:00






          • 1




            @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
            – Kusalananda
            Jul 11 at 13:38










          Your Answer







          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "106"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: false,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );








           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f454687%2fpass-a-list-of-directories-to-rsync%23new-answer', 'question_page');

          );

          Post as a guest






























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          2
          down vote













          rsync -av ...options... --files-from=list_of_dirs_to_copy 
          /mnt/storage3/workfiles/backups/
          /drives/f/data_backups


          The paths in list_of_dirs_to_copy would be interpreted as being relative to the source directory. With the given list, this would create data as a subdirectory under /drives/f/data_backups.






          share|improve this answer





















          • This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
            – benett
            Jul 11 at 13:00










          • The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
            – benett
            Jul 11 at 13:00






          • 1




            @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
            – Kusalananda
            Jul 11 at 13:38














          up vote
          2
          down vote













          rsync -av ...options... --files-from=list_of_dirs_to_copy 
          /mnt/storage3/workfiles/backups/
          /drives/f/data_backups


          The paths in list_of_dirs_to_copy would be interpreted as being relative to the source directory. With the given list, this would create data as a subdirectory under /drives/f/data_backups.






          share|improve this answer





















          • This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
            – benett
            Jul 11 at 13:00










          • The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
            – benett
            Jul 11 at 13:00






          • 1




            @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
            – Kusalananda
            Jul 11 at 13:38












          up vote
          2
          down vote










          up vote
          2
          down vote









          rsync -av ...options... --files-from=list_of_dirs_to_copy 
          /mnt/storage3/workfiles/backups/
          /drives/f/data_backups


          The paths in list_of_dirs_to_copy would be interpreted as being relative to the source directory. With the given list, this would create data as a subdirectory under /drives/f/data_backups.






          share|improve this answer













          rsync -av ...options... --files-from=list_of_dirs_to_copy 
          /mnt/storage3/workfiles/backups/
          /drives/f/data_backups


          The paths in list_of_dirs_to_copy would be interpreted as being relative to the source directory. With the given list, this would create data as a subdirectory under /drives/f/data_backups.







          share|improve this answer













          share|improve this answer



          share|improve this answer











          answered Jul 11 at 12:35









          Kusalananda

          101k13199312




          101k13199312











          • This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
            – benett
            Jul 11 at 13:00










          • The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
            – benett
            Jul 11 at 13:00






          • 1




            @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
            – Kusalananda
            Jul 11 at 13:38
















          • This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
            – benett
            Jul 11 at 13:00










          • The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
            – benett
            Jul 11 at 13:00






          • 1




            @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
            – Kusalananda
            Jul 11 at 13:38















          This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
          – benett
          Jul 11 at 13:00




          This is most probably caused by my rsync (I am using MobaXterm) which leads me to an error when I use the --files-from tag. So I instead opted for: while read -r COPYPATHS; do rsync -R -rvh --append-verify --chmod=ugo=rwX --no-o --no-g --no-perms --progress --stats "$COPYPATHS" /drives/f/data_backups; done < /home/files/list_of_dirs_to_copy.txt. This no longer produces an error.
          – benett
          Jul 11 at 13:00












          The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
          – benett
          Jul 11 at 13:00




          The error I am receiving is: Use "rsync --daemon --help" to see the daemon-mode command-line options. Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation. See http://rsync.samba.org/ for updates, bug reports, and answers rsync error: syntax or usage error (code 1) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/options.c(1653) [client=3.0.9]
          – benett
          Jul 11 at 13:00




          1




          1




          @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
          – Kusalananda
          Jul 11 at 13:38




          @benett Huh? I hope you replaced my "...options..." placeholder with your actual options.
          – Kusalananda
          Jul 11 at 13:38












           

          draft saved


          draft discarded


























           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f454687%2fpass-a-list-of-directories-to-rsync%23new-answer', 'question_page');

          );

          Post as a guest













































































          Popular posts from this blog

          How to check contact read email or not when send email to Individual?

          Bahrain

          Postfix configuration issue with fips on centos 7; mailgun relay