Sort and copy files from several directories

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite












I have a directory containing several sub-directories, containing files.



I would like to sort the files in each of these directories by name alphabetically and copy the first file from each directory to a given new directory.



  • I would like to ignore hidden files.

  • If there is a file with the same name in the destination directory, both files should be kept.

Sample:



parentdir
->dir1
->afile
->bfile
->cfile
->dir2
->dfile
->efile
->ffile
->dir3
->afile
->hfile
->ifile


I would like to run a command that would populate my new directory like so:



newdir
->afile
->afile.1
->dfile


How can this be accomplished?







share|improve this question






















  • How do you define 'first' then?
    – ilkkachu
    Dec 24 '17 at 13:16










  • i know there are some examples on how to move first x amount of files. i am just figuring linux internal mechanics could sort that out?. thought about a modified version from this #find . -maxdepth 1 -type f |head -1000|xargs cp -t "$destdir"
    – dekatch
    Dec 24 '17 at 13:21







  • 1




    Please edit your question and explain what you mean by "first". First by date? First by alphabetical order? Just any random file? What about hidden files? Do they count? Is the new destination a directory? What if you have files with the same name? Should they be overwritten?
    – terdon♦
    Dec 24 '17 at 13:25










  • ok, done terdon. whew.. dunno if i still feel comfortable with that requests ! i think i ask for to much xx
    – dekatch
    Dec 24 '17 at 13:37














up vote
0
down vote

favorite












I have a directory containing several sub-directories, containing files.



I would like to sort the files in each of these directories by name alphabetically and copy the first file from each directory to a given new directory.



  • I would like to ignore hidden files.

  • If there is a file with the same name in the destination directory, both files should be kept.

Sample:



parentdir
->dir1
->afile
->bfile
->cfile
->dir2
->dfile
->efile
->ffile
->dir3
->afile
->hfile
->ifile


I would like to run a command that would populate my new directory like so:



newdir
->afile
->afile.1
->dfile


How can this be accomplished?







share|improve this question






















  • How do you define 'first' then?
    – ilkkachu
    Dec 24 '17 at 13:16










  • i know there are some examples on how to move first x amount of files. i am just figuring linux internal mechanics could sort that out?. thought about a modified version from this #find . -maxdepth 1 -type f |head -1000|xargs cp -t "$destdir"
    – dekatch
    Dec 24 '17 at 13:21







  • 1




    Please edit your question and explain what you mean by "first". First by date? First by alphabetical order? Just any random file? What about hidden files? Do they count? Is the new destination a directory? What if you have files with the same name? Should they be overwritten?
    – terdon♦
    Dec 24 '17 at 13:25










  • ok, done terdon. whew.. dunno if i still feel comfortable with that requests ! i think i ask for to much xx
    – dekatch
    Dec 24 '17 at 13:37












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have a directory containing several sub-directories, containing files.



I would like to sort the files in each of these directories by name alphabetically and copy the first file from each directory to a given new directory.



  • I would like to ignore hidden files.

  • If there is a file with the same name in the destination directory, both files should be kept.

Sample:



parentdir
->dir1
->afile
->bfile
->cfile
->dir2
->dfile
->efile
->ffile
->dir3
->afile
->hfile
->ifile


I would like to run a command that would populate my new directory like so:



newdir
->afile
->afile.1
->dfile


How can this be accomplished?







share|improve this question














I have a directory containing several sub-directories, containing files.



I would like to sort the files in each of these directories by name alphabetically and copy the first file from each directory to a given new directory.



  • I would like to ignore hidden files.

  • If there is a file with the same name in the destination directory, both files should be kept.

Sample:



parentdir
->dir1
->afile
->bfile
->cfile
->dir2
->dfile
->efile
->ffile
->dir3
->afile
->hfile
->ifile


I would like to run a command that would populate my new directory like so:



newdir
->afile
->afile.1
->dfile


How can this be accomplished?









share|improve this question













share|improve this question




share|improve this question








edited Dec 24 '17 at 14:15









Jesse_b

10.5k22659




10.5k22659










asked Dec 24 '17 at 13:14









dekatch

11




11











  • How do you define 'first' then?
    – ilkkachu
    Dec 24 '17 at 13:16










  • i know there are some examples on how to move first x amount of files. i am just figuring linux internal mechanics could sort that out?. thought about a modified version from this #find . -maxdepth 1 -type f |head -1000|xargs cp -t "$destdir"
    – dekatch
    Dec 24 '17 at 13:21







  • 1




    Please edit your question and explain what you mean by "first". First by date? First by alphabetical order? Just any random file? What about hidden files? Do they count? Is the new destination a directory? What if you have files with the same name? Should they be overwritten?
    – terdon♦
    Dec 24 '17 at 13:25










  • ok, done terdon. whew.. dunno if i still feel comfortable with that requests ! i think i ask for to much xx
    – dekatch
    Dec 24 '17 at 13:37
















  • How do you define 'first' then?
    – ilkkachu
    Dec 24 '17 at 13:16










  • i know there are some examples on how to move first x amount of files. i am just figuring linux internal mechanics could sort that out?. thought about a modified version from this #find . -maxdepth 1 -type f |head -1000|xargs cp -t "$destdir"
    – dekatch
    Dec 24 '17 at 13:21







  • 1




    Please edit your question and explain what you mean by "first". First by date? First by alphabetical order? Just any random file? What about hidden files? Do they count? Is the new destination a directory? What if you have files with the same name? Should they be overwritten?
    – terdon♦
    Dec 24 '17 at 13:25










  • ok, done terdon. whew.. dunno if i still feel comfortable with that requests ! i think i ask for to much xx
    – dekatch
    Dec 24 '17 at 13:37















How do you define 'first' then?
– ilkkachu
Dec 24 '17 at 13:16




How do you define 'first' then?
– ilkkachu
Dec 24 '17 at 13:16












i know there are some examples on how to move first x amount of files. i am just figuring linux internal mechanics could sort that out?. thought about a modified version from this #find . -maxdepth 1 -type f |head -1000|xargs cp -t "$destdir"
– dekatch
Dec 24 '17 at 13:21





i know there are some examples on how to move first x amount of files. i am just figuring linux internal mechanics could sort that out?. thought about a modified version from this #find . -maxdepth 1 -type f |head -1000|xargs cp -t "$destdir"
– dekatch
Dec 24 '17 at 13:21





1




1




Please edit your question and explain what you mean by "first". First by date? First by alphabetical order? Just any random file? What about hidden files? Do they count? Is the new destination a directory? What if you have files with the same name? Should they be overwritten?
– terdon♦
Dec 24 '17 at 13:25




Please edit your question and explain what you mean by "first". First by date? First by alphabetical order? Just any random file? What about hidden files? Do they count? Is the new destination a directory? What if you have files with the same name? Should they be overwritten?
– terdon♦
Dec 24 '17 at 13:25












ok, done terdon. whew.. dunno if i still feel comfortable with that requests ! i think i ask for to much xx
– dekatch
Dec 24 '17 at 13:37




ok, done terdon. whew.. dunno if i still feel comfortable with that requests ! i think i ask for to much xx
– dekatch
Dec 24 '17 at 13:37










3 Answers
3






active

oldest

votes

















up vote
0
down vote













I guess the straightforward way is to use ls to list the files, which it does in alphabetical order (and omitting hidden files) by default, take the first one from each directory with head, and then have a bit of shell code to figure out what filename they should be copied to. As in this bash script:



for d in ./*/; do
src="$(ls "$d" | head -n 1)"
dstbase="newdir/$(basename "$src")"
if [[ ! -f "$dstbase" ]]; then
cp "$src" "$dstbase"
else
suffix=1
while [[ -f "$dstbase.$suffix" ]]; do
let suffix++
done
cp "$src" "$dstbase.$suffix"
fi
done


Obviously you will have to adjust the file paths as needed for your actual situation. This is also fairly fragile in the sense that it may have problems if there are any directories under dir1, dir2, etc. which you would want to exclude from being copied. It also has a potential race condition in the loop that looks for an unused filename; that probably won't be a problem, but don't do anything like running multiple copies of this in parallel.






share|improve this answer



























    up vote
    0
    down vote













    This can be achieved with the following one line:



    ls -ltr /parent_directory_path/ |
    awk '$1~"^drw" print $9' |
    while read line; do
    ls -l $line |
    awk '$1 ~ "^-r" print $9' |
    head -1 |
    awk -v line="$line" 'print "cp -rvfp" " " line"/"$1 " " "/newdir/" ';
    done |
    sh





    share|improve this answer





























      up vote
      0
      down vote













      With zsh and GNU cp:



      cd parentdir &&
      for dir (*(/)) (cp --backup=numbered -- $dir/*(.[1]) newdir/)


      the --backup=numbered implements a backup scheme where files are named file.~1~, file.~2~... So the first copy will make will be file.~max~ and the last one file (process the list backward (*(/On) instead of *(/)) if you want the order reversed).



      POSIXly, you could do:



      cp_numbered() (
      suffix=
      n=0
      while [ -e "$2$suffix" ] || [ -L "$2$suffix" ]; do
      suffix=.$((n += 1))
      done
      exec cp -- "$1" "$2$suffix"
      )
      cd parentdir &&
      for dir in */; do
      for file in "$dir"/*; do
      if [ -f "$file" ]; then
      cp_numbered "$file" "newdir/$file#*/"
      continue 2
      fi
      done
      done





      share|improve this answer






















        Your Answer







        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "106"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        convertImagesToLinks: false,
        noModals: false,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: null,
        bindNavPrevention: true,
        postfix: "",
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );








         

        draft saved


        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f412804%2fsort-and-copy-files-from-several-directories%23new-answer', 'question_page');

        );

        Post as a guest






























        3 Answers
        3






        active

        oldest

        votes








        3 Answers
        3






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes








        up vote
        0
        down vote













        I guess the straightforward way is to use ls to list the files, which it does in alphabetical order (and omitting hidden files) by default, take the first one from each directory with head, and then have a bit of shell code to figure out what filename they should be copied to. As in this bash script:



        for d in ./*/; do
        src="$(ls "$d" | head -n 1)"
        dstbase="newdir/$(basename "$src")"
        if [[ ! -f "$dstbase" ]]; then
        cp "$src" "$dstbase"
        else
        suffix=1
        while [[ -f "$dstbase.$suffix" ]]; do
        let suffix++
        done
        cp "$src" "$dstbase.$suffix"
        fi
        done


        Obviously you will have to adjust the file paths as needed for your actual situation. This is also fairly fragile in the sense that it may have problems if there are any directories under dir1, dir2, etc. which you would want to exclude from being copied. It also has a potential race condition in the loop that looks for an unused filename; that probably won't be a problem, but don't do anything like running multiple copies of this in parallel.






        share|improve this answer
























          up vote
          0
          down vote













          I guess the straightforward way is to use ls to list the files, which it does in alphabetical order (and omitting hidden files) by default, take the first one from each directory with head, and then have a bit of shell code to figure out what filename they should be copied to. As in this bash script:



          for d in ./*/; do
          src="$(ls "$d" | head -n 1)"
          dstbase="newdir/$(basename "$src")"
          if [[ ! -f "$dstbase" ]]; then
          cp "$src" "$dstbase"
          else
          suffix=1
          while [[ -f "$dstbase.$suffix" ]]; do
          let suffix++
          done
          cp "$src" "$dstbase.$suffix"
          fi
          done


          Obviously you will have to adjust the file paths as needed for your actual situation. This is also fairly fragile in the sense that it may have problems if there are any directories under dir1, dir2, etc. which you would want to exclude from being copied. It also has a potential race condition in the loop that looks for an unused filename; that probably won't be a problem, but don't do anything like running multiple copies of this in parallel.






          share|improve this answer






















            up vote
            0
            down vote










            up vote
            0
            down vote









            I guess the straightforward way is to use ls to list the files, which it does in alphabetical order (and omitting hidden files) by default, take the first one from each directory with head, and then have a bit of shell code to figure out what filename they should be copied to. As in this bash script:



            for d in ./*/; do
            src="$(ls "$d" | head -n 1)"
            dstbase="newdir/$(basename "$src")"
            if [[ ! -f "$dstbase" ]]; then
            cp "$src" "$dstbase"
            else
            suffix=1
            while [[ -f "$dstbase.$suffix" ]]; do
            let suffix++
            done
            cp "$src" "$dstbase.$suffix"
            fi
            done


            Obviously you will have to adjust the file paths as needed for your actual situation. This is also fairly fragile in the sense that it may have problems if there are any directories under dir1, dir2, etc. which you would want to exclude from being copied. It also has a potential race condition in the loop that looks for an unused filename; that probably won't be a problem, but don't do anything like running multiple copies of this in parallel.






            share|improve this answer












            I guess the straightforward way is to use ls to list the files, which it does in alphabetical order (and omitting hidden files) by default, take the first one from each directory with head, and then have a bit of shell code to figure out what filename they should be copied to. As in this bash script:



            for d in ./*/; do
            src="$(ls "$d" | head -n 1)"
            dstbase="newdir/$(basename "$src")"
            if [[ ! -f "$dstbase" ]]; then
            cp "$src" "$dstbase"
            else
            suffix=1
            while [[ -f "$dstbase.$suffix" ]]; do
            let suffix++
            done
            cp "$src" "$dstbase.$suffix"
            fi
            done


            Obviously you will have to adjust the file paths as needed for your actual situation. This is also fairly fragile in the sense that it may have problems if there are any directories under dir1, dir2, etc. which you would want to exclude from being copied. It also has a potential race condition in the loop that looks for an unused filename; that probably won't be a problem, but don't do anything like running multiple copies of this in parallel.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Dec 24 '17 at 23:04









            David Z

            555310




            555310






















                up vote
                0
                down vote













                This can be achieved with the following one line:



                ls -ltr /parent_directory_path/ |
                awk '$1~"^drw" print $9' |
                while read line; do
                ls -l $line |
                awk '$1 ~ "^-r" print $9' |
                head -1 |
                awk -v line="$line" 'print "cp -rvfp" " " line"/"$1 " " "/newdir/" ';
                done |
                sh





                share|improve this answer


























                  up vote
                  0
                  down vote













                  This can be achieved with the following one line:



                  ls -ltr /parent_directory_path/ |
                  awk '$1~"^drw" print $9' |
                  while read line; do
                  ls -l $line |
                  awk '$1 ~ "^-r" print $9' |
                  head -1 |
                  awk -v line="$line" 'print "cp -rvfp" " " line"/"$1 " " "/newdir/" ';
                  done |
                  sh





                  share|improve this answer
























                    up vote
                    0
                    down vote










                    up vote
                    0
                    down vote









                    This can be achieved with the following one line:



                    ls -ltr /parent_directory_path/ |
                    awk '$1~"^drw" print $9' |
                    while read line; do
                    ls -l $line |
                    awk '$1 ~ "^-r" print $9' |
                    head -1 |
                    awk -v line="$line" 'print "cp -rvfp" " " line"/"$1 " " "/newdir/" ';
                    done |
                    sh





                    share|improve this answer














                    This can be achieved with the following one line:



                    ls -ltr /parent_directory_path/ |
                    awk '$1~"^drw" print $9' |
                    while read line; do
                    ls -l $line |
                    awk '$1 ~ "^-r" print $9' |
                    head -1 |
                    awk -v line="$line" 'print "cp -rvfp" " " line"/"$1 " " "/newdir/" ';
                    done |
                    sh






                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited Jan 5 at 13:40









                    grg

                    1857




                    1857










                    answered Dec 25 '17 at 6:58









                    Praveen Kumar BS

                    1,010128




                    1,010128




















                        up vote
                        0
                        down vote













                        With zsh and GNU cp:



                        cd parentdir &&
                        for dir (*(/)) (cp --backup=numbered -- $dir/*(.[1]) newdir/)


                        the --backup=numbered implements a backup scheme where files are named file.~1~, file.~2~... So the first copy will make will be file.~max~ and the last one file (process the list backward (*(/On) instead of *(/)) if you want the order reversed).



                        POSIXly, you could do:



                        cp_numbered() (
                        suffix=
                        n=0
                        while [ -e "$2$suffix" ] || [ -L "$2$suffix" ]; do
                        suffix=.$((n += 1))
                        done
                        exec cp -- "$1" "$2$suffix"
                        )
                        cd parentdir &&
                        for dir in */; do
                        for file in "$dir"/*; do
                        if [ -f "$file" ]; then
                        cp_numbered "$file" "newdir/$file#*/"
                        continue 2
                        fi
                        done
                        done





                        share|improve this answer


























                          up vote
                          0
                          down vote













                          With zsh and GNU cp:



                          cd parentdir &&
                          for dir (*(/)) (cp --backup=numbered -- $dir/*(.[1]) newdir/)


                          the --backup=numbered implements a backup scheme where files are named file.~1~, file.~2~... So the first copy will make will be file.~max~ and the last one file (process the list backward (*(/On) instead of *(/)) if you want the order reversed).



                          POSIXly, you could do:



                          cp_numbered() (
                          suffix=
                          n=0
                          while [ -e "$2$suffix" ] || [ -L "$2$suffix" ]; do
                          suffix=.$((n += 1))
                          done
                          exec cp -- "$1" "$2$suffix"
                          )
                          cd parentdir &&
                          for dir in */; do
                          for file in "$dir"/*; do
                          if [ -f "$file" ]; then
                          cp_numbered "$file" "newdir/$file#*/"
                          continue 2
                          fi
                          done
                          done





                          share|improve this answer
























                            up vote
                            0
                            down vote










                            up vote
                            0
                            down vote









                            With zsh and GNU cp:



                            cd parentdir &&
                            for dir (*(/)) (cp --backup=numbered -- $dir/*(.[1]) newdir/)


                            the --backup=numbered implements a backup scheme where files are named file.~1~, file.~2~... So the first copy will make will be file.~max~ and the last one file (process the list backward (*(/On) instead of *(/)) if you want the order reversed).



                            POSIXly, you could do:



                            cp_numbered() (
                            suffix=
                            n=0
                            while [ -e "$2$suffix" ] || [ -L "$2$suffix" ]; do
                            suffix=.$((n += 1))
                            done
                            exec cp -- "$1" "$2$suffix"
                            )
                            cd parentdir &&
                            for dir in */; do
                            for file in "$dir"/*; do
                            if [ -f "$file" ]; then
                            cp_numbered "$file" "newdir/$file#*/"
                            continue 2
                            fi
                            done
                            done





                            share|improve this answer














                            With zsh and GNU cp:



                            cd parentdir &&
                            for dir (*(/)) (cp --backup=numbered -- $dir/*(.[1]) newdir/)


                            the --backup=numbered implements a backup scheme where files are named file.~1~, file.~2~... So the first copy will make will be file.~max~ and the last one file (process the list backward (*(/On) instead of *(/)) if you want the order reversed).



                            POSIXly, you could do:



                            cp_numbered() (
                            suffix=
                            n=0
                            while [ -e "$2$suffix" ] || [ -L "$2$suffix" ]; do
                            suffix=.$((n += 1))
                            done
                            exec cp -- "$1" "$2$suffix"
                            )
                            cd parentdir &&
                            for dir in */; do
                            for file in "$dir"/*; do
                            if [ -f "$file" ]; then
                            cp_numbered "$file" "newdir/$file#*/"
                            continue 2
                            fi
                            done
                            done






                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited Jan 5 at 14:16

























                            answered Jan 5 at 14:02









                            Stéphane Chazelas

                            282k53518851




                            282k53518851






















                                 

                                draft saved


                                draft discarded


























                                 


                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f412804%2fsort-and-copy-files-from-several-directories%23new-answer', 'question_page');

                                );

                                Post as a guest













































































                                Popular posts from this blog

                                How to check contact read email or not when send email to Individual?

                                Bahrain

                                Postfix configuration issue with fips on centos 7; mailgun relay