How to create a recursive manifest directory

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx









share|improve this question



















  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    Dec 7 at 6:53










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    Dec 7 at 17:32










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    Dec 9 at 15:02














up vote
1
down vote

favorite












I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx









share|improve this question



















  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    Dec 7 at 6:53










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    Dec 7 at 17:32










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    Dec 9 at 15:02












up vote
1
down vote

favorite









up vote
1
down vote

favorite











I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx









share|improve this question















I need to create a manifest of thousands of files and their directory paths recursively.



Here is an example of how i need the manifest to output



This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav

This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav


the command tree -fai > manifest.txt gets me close to what i need but it does not create a line break after the absolute path.



secondly i would like to output sequential files in a subdirectory as 1 single line input for example



This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx


displayed as below instead



 This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx






linux ls






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Dec 9 at 15:01

























asked Dec 7 at 6:00









ioshifting

62




62







  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    Dec 7 at 6:53










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    Dec 7 at 17:32










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    Dec 9 at 15:02












  • 1




    In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
    – Kusalananda
    Dec 7 at 6:53










  • ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
    – K7AAY
    Dec 7 at 17:32










  • Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
    – ioshifting
    Dec 9 at 15:02







1




1




In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
Dec 7 at 6:53




In the second part, the numbers in given pattern, test_file_here_[0000000-00001234].dpx, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
Dec 7 at 6:53












ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32




ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32












Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02




Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02










3 Answers
3






active

oldest

votes

















up vote
1
down vote













From man:




 -R, --recursive
list subdirectories recursively



Try,



ls -R /path/to/dir


more closely,



ll -R /path/to/dir | awk '$1!="total"print $NF'





share|improve this answer




















  • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
    – ioshifting
    Dec 9 at 15:10










  • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
    – ioshifting
    Dec 9 at 15:56

















up vote
0
down vote













find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





share|improve this answer



























    up vote
    0
    down vote













    Since you want to find directories and then list their contents, why not use find for it:



    find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
    for pathname do
    header=0
    for filename in "$pathname"/*; do
    if [ ! -d "$filename" ]; then
    if [ "$header" -eq 0 ]; then
    printf "%s/n" "$pathname%/"
    header=1
    fi
    printf "%sn" "$filename##*/"
    fi
    done
    [ "$header" -eq 1 ] && printf "n"
    done' bash +


    This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



    The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



    Directories that are empty or that only contains directories are not listed.



    For a directory structure such as



    /dir
    `-- a
    |-- b
    | `-- c
    | |-- .hidden_file
    | `-- file
    `-- file


    This would produce the following output:



    /dir/a/
    file

    /dir/a/b/c/
    .hidden_file
    file





    share|improve this answer




















    • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
      – ioshifting
      Dec 9 at 15:15










    • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
      – Kusalananda
      Dec 9 at 23:11










    Your Answer








    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote













    From man:




     -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"print $NF'





    share|improve this answer




















    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      Dec 9 at 15:10










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      Dec 9 at 15:56














    up vote
    1
    down vote













    From man:




     -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"print $NF'





    share|improve this answer




















    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      Dec 9 at 15:10










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      Dec 9 at 15:56












    up vote
    1
    down vote










    up vote
    1
    down vote









    From man:




     -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"print $NF'





    share|improve this answer












    From man:




     -R, --recursive
    list subdirectories recursively



    Try,



    ls -R /path/to/dir


    more closely,



    ll -R /path/to/dir | awk '$1!="total"print $NF'






    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Dec 7 at 6:12









    msp9011

    3,65543863




    3,65543863











    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      Dec 9 at 15:10










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      Dec 9 at 15:56
















    • the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
      – ioshifting
      Dec 9 at 15:10










    • do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
      – ioshifting
      Dec 9 at 15:56















    the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
    – ioshifting
    Dec 9 at 15:10




    the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
    – ioshifting
    Dec 9 at 15:10












    do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
    – ioshifting
    Dec 9 at 15:56




    do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
    – ioshifting
    Dec 9 at 15:56












    up vote
    0
    down vote













    find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





    share|improve this answer
























      up vote
      0
      down vote













      find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





      share|improve this answer






















        up vote
        0
        down vote










        up vote
        0
        down vote









        find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'





        share|improve this answer












        find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Dec 8 at 0:30









        Ole Tange

        11.9k1451105




        11.9k1451105




















            up vote
            0
            down vote













            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "$pathname%/"
            header=1
            fi
            printf "%sn" "$filename##*/"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file





            share|improve this answer




















            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              Dec 9 at 15:15










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              Dec 9 at 23:11














            up vote
            0
            down vote













            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "$pathname%/"
            header=1
            fi
            printf "%sn" "$filename##*/"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file





            share|improve this answer




















            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              Dec 9 at 15:15










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              Dec 9 at 23:11












            up vote
            0
            down vote










            up vote
            0
            down vote









            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "$pathname%/"
            header=1
            fi
            printf "%sn" "$filename##*/"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file





            share|improve this answer












            Since you want to find directories and then list their contents, why not use find for it:



            find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
            for pathname do
            header=0
            for filename in "$pathname"/*; do
            if [ ! -d "$filename" ]; then
            if [ "$header" -eq 0 ]; then
            printf "%s/n" "$pathname%/"
            header=1
            fi
            printf "%sn" "$filename##*/"
            fi
            done
            [ "$header" -eq 1 ] && printf "n"
            done' bash +


            This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash script.



            The bash script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.



            Directories that are empty or that only contains directories are not listed.



            For a directory structure such as



            /dir
            `-- a
            |-- b
            | `-- c
            | |-- .hidden_file
            | `-- file
            `-- file


            This would produce the following output:



            /dir/a/
            file

            /dir/a/b/c/
            .hidden_file
            file






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Dec 8 at 9:05









            Kusalananda

            120k16225369




            120k16225369











            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              Dec 9 at 15:15










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              Dec 9 at 23:11
















            • This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
              – ioshifting
              Dec 9 at 15:15










            • @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
              – Kusalananda
              Dec 9 at 23:11















            This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
            – ioshifting
            Dec 9 at 15:15




            This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
            – ioshifting
            Dec 9 at 15:15












            @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
            – Kusalananda
            Dec 9 at 23:11




            @ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
            – Kusalananda
            Dec 9 at 23:11

















            draft saved

            draft discarded
















































            Thanks for contributing an answer to Unix & Linux Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown






            Popular posts from this blog

            How to check contact read email or not when send email to Individual?

            Displaying single band from multi-band raster using QGIS

            How many registers does an x86_64 CPU actually have?