How to create a recursive manifest directory
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
add a comment |
up vote
1
down vote
favorite
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
1
In the second part, the numbers in given pattern,test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
Dec 7 at 6:53
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
I need to create a manifest of thousands of files and their directory paths recursively.
Here is an example of how i need the manifest to output
This_is_an_example/of_how/i_want_to_display/absolute_paths/
examplefiles.md5
examplefiles.txt
examplefile.wav
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_2/
examplefiles.md5
examplefiles.txt
examplefile.wav
the command tree -fai > manifest.txt
gets me close to what i need but it does not create a line break after the absolute path.
secondly i would like to output sequential files in a subdirectory as 1 single line input for example
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_3/
test_file_here_0000001.dpx
test_file_here_0000002.dpx
test_file_here_0000003.dpx
test_file_here_0000004.dpx
displayed as below instead
This_is_an_example/of_how/i_want_to_display/absolute_paths/part_4/
test_file_here_[0000000-0000004].dpx
linux ls
linux ls
edited Dec 9 at 15:01
asked Dec 7 at 6:00
ioshifting
62
62
1
In the second part, the numbers in given pattern,test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
Dec 7 at 6:53
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02
add a comment |
1
In the second part, the numbers in given pattern,test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.
– Kusalananda
Dec 7 at 6:53
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02
1
1
In the second part, the numbers in given pattern,
test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.– Kusalananda
Dec 7 at 6:53
In the second part, the numbers in given pattern,
test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.– Kusalananda
Dec 7 at 6:53
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02
add a comment |
3 Answers
3
active
oldest
votes
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"print $NF'
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
add a comment |
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
add a comment |
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "$pathname%/"
header=1
fi
printf "%sn" "$filename##*/"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"print $NF'
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
add a comment |
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"print $NF'
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
add a comment |
up vote
1
down vote
up vote
1
down vote
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"print $NF'
From man
:
-R, --recursive
list subdirectories recursively
Try,
ls -R /path/to/dir
more closely,
ll -R /path/to/dir | awk '$1!="total"print $NF'
answered Dec 7 at 6:12
msp9011
3,65543863
3,65543863
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
add a comment |
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
the second example with the awk option prints all the files and folder paths how i need them thank you! - i just need to find out how to edit the sequential files into a singular input when printed out
– ioshifting
Dec 9 at 15:10
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
do you perhaps know the syntax for the awk part of the script to work on centos? i have gawk installed.
– ioshifting
Dec 9 at 15:56
add a comment |
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
add a comment |
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
add a comment |
up vote
0
down vote
up vote
0
down vote
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
find . -print | perl -pe 'a=$_; chomp $a; -d $a or s:.*/::'
answered Dec 8 at 0:30
Ole Tange
11.9k1451105
11.9k1451105
add a comment |
add a comment |
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "$pathname%/"
header=1
fi
printf "%sn" "$filename##*/"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
add a comment |
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "$pathname%/"
header=1
fi
printf "%sn" "$filename##*/"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
add a comment |
up vote
0
down vote
up vote
0
down vote
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "$pathname%/"
header=1
fi
printf "%sn" "$filename##*/"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
Since you want to find directories and then list their contents, why not use find
for it:
find /path/to/dir -type d -exec bash -O dotglob -O nullglob -c '
for pathname do
header=0
for filename in "$pathname"/*; do
if [ ! -d "$filename" ]; then
if [ "$header" -eq 0 ]; then
printf "%s/n" "$pathname%/"
header=1
fi
printf "%sn" "$filename##*/"
fi
done
[ "$header" -eq 1 ] && printf "n"
done' bash +
This finds all directories in or under the given directory path. It feeds the pathnames of all these directories to a bash
script.
The bash
script will iterate over each directory, printing the directory pathname as a header and listing non-directory entries present in that directory. At the end, if at least one non-directory file was found in the directory, an extra newline is outputted.
Directories that are empty or that only contains directories are not listed.
For a directory structure such as
/dir
`-- a
|-- b
| `-- c
| |-- .hidden_file
| `-- file
`-- file
This would produce the following output:
/dir/a/
file
/dir/a/b/c/
.hidden_file
file
answered Dec 8 at 9:05
Kusalananda
120k16225369
120k16225369
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
add a comment |
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
This works very well and displays all hidden files - however i did notice that a lot of the files in the subdirectories appeared to duplicate for example test_file.txt would also have a seperate file labelled ._test_file.txt in some instances whole directories were duplicated.
– ioshifting
Dec 9 at 15:15
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
@ioshifting I don't think that's a sid effect of using my code. Could you check to see if those hidden and seemingly duplicate file are actually there, and what they are?
– Kusalananda
Dec 9 at 23:11
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f486507%2fhow-to-create-a-recursive-manifest-directory%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
In the second part, the numbers in given pattern,
test_file_here_[0000000-00001234].dpx
, bears no resemblance to the numbers in the filenames shown.– Kusalananda
Dec 7 at 6:53
ioshifting, would you please clarify your original post by clicking on edit and expanding on the last example? Folks trying to help here use Comments, but we want you to edit the original post so all can see the change requested.
– K7AAY
Dec 7 at 17:32
Sorry both, i have now corrected the example given to correctly match what i am looking to achieve
– ioshifting
Dec 9 at 15:02