Running Python script over multiple directories and multiple files

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP












0















Here is my command line for python:



python script.py -a .faa -n .fna


I have a directory containing multiple directories. Out of these multiple directories, there could be 1 or more folders.



In these 1 or more folders, I need to run my python script on the .faa and .fna files.



How do I do this?



Example tree:



Staph1 ----> [CO1, CO2] ----> [.fna, .faa] from CO1, [.fna, .faa] from CO2

Staph2 ----> [CO6] ----> [.fna, .faa] from CO6


My Python script will use the inputs .fna and .faa and output within the folder. Also the folder and directory are synonymous.










share|improve this question



















  • 3





    Did you code script.py? Why not use os.walk to navigate the directory from within script.py?

    – iruvar
    Aug 14 '14 at 20:23












  • There are over 4000 genomes, each containing 1 or more contigs. would have to use my command line 4000+ times just to get the files i need.

    – Jeff
    Aug 14 '14 at 20:38






  • 1





    You use both folder and directory in your question. Is there a difference between those two things? If so, what is the difference? Also, please edit your question to include a portion of an example directory structure for us to look at.

    – dg99
    Aug 14 '14 at 20:46












  • A tree representing the directory hierarchy would be nice.

    – Cristian Ciupitu
    Aug 14 '14 at 20:56











  • @Jeff does script.py takes file names or directories? Do .fna and .faa come in pairs?

    – Simply_Me
    Aug 14 '14 at 21:48















0















Here is my command line for python:



python script.py -a .faa -n .fna


I have a directory containing multiple directories. Out of these multiple directories, there could be 1 or more folders.



In these 1 or more folders, I need to run my python script on the .faa and .fna files.



How do I do this?



Example tree:



Staph1 ----> [CO1, CO2] ----> [.fna, .faa] from CO1, [.fna, .faa] from CO2

Staph2 ----> [CO6] ----> [.fna, .faa] from CO6


My Python script will use the inputs .fna and .faa and output within the folder. Also the folder and directory are synonymous.










share|improve this question



















  • 3





    Did you code script.py? Why not use os.walk to navigate the directory from within script.py?

    – iruvar
    Aug 14 '14 at 20:23












  • There are over 4000 genomes, each containing 1 or more contigs. would have to use my command line 4000+ times just to get the files i need.

    – Jeff
    Aug 14 '14 at 20:38






  • 1





    You use both folder and directory in your question. Is there a difference between those two things? If so, what is the difference? Also, please edit your question to include a portion of an example directory structure for us to look at.

    – dg99
    Aug 14 '14 at 20:46












  • A tree representing the directory hierarchy would be nice.

    – Cristian Ciupitu
    Aug 14 '14 at 20:56











  • @Jeff does script.py takes file names or directories? Do .fna and .faa come in pairs?

    – Simply_Me
    Aug 14 '14 at 21:48













0












0








0


0






Here is my command line for python:



python script.py -a .faa -n .fna


I have a directory containing multiple directories. Out of these multiple directories, there could be 1 or more folders.



In these 1 or more folders, I need to run my python script on the .faa and .fna files.



How do I do this?



Example tree:



Staph1 ----> [CO1, CO2] ----> [.fna, .faa] from CO1, [.fna, .faa] from CO2

Staph2 ----> [CO6] ----> [.fna, .faa] from CO6


My Python script will use the inputs .fna and .faa and output within the folder. Also the folder and directory are synonymous.










share|improve this question
















Here is my command line for python:



python script.py -a .faa -n .fna


I have a directory containing multiple directories. Out of these multiple directories, there could be 1 or more folders.



In these 1 or more folders, I need to run my python script on the .faa and .fna files.



How do I do this?



Example tree:



Staph1 ----> [CO1, CO2] ----> [.fna, .faa] from CO1, [.fna, .faa] from CO2

Staph2 ----> [CO6] ----> [.fna, .faa] from CO6


My Python script will use the inputs .fna and .faa and output within the folder. Also the folder and directory are synonymous.







bash python






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Dec 16 '18 at 11:44









Rui F Ribeiro

40.1k1479136




40.1k1479136










asked Aug 14 '14 at 20:20









JeffJeff

12




12







  • 3





    Did you code script.py? Why not use os.walk to navigate the directory from within script.py?

    – iruvar
    Aug 14 '14 at 20:23












  • There are over 4000 genomes, each containing 1 or more contigs. would have to use my command line 4000+ times just to get the files i need.

    – Jeff
    Aug 14 '14 at 20:38






  • 1





    You use both folder and directory in your question. Is there a difference between those two things? If so, what is the difference? Also, please edit your question to include a portion of an example directory structure for us to look at.

    – dg99
    Aug 14 '14 at 20:46












  • A tree representing the directory hierarchy would be nice.

    – Cristian Ciupitu
    Aug 14 '14 at 20:56











  • @Jeff does script.py takes file names or directories? Do .fna and .faa come in pairs?

    – Simply_Me
    Aug 14 '14 at 21:48












  • 3





    Did you code script.py? Why not use os.walk to navigate the directory from within script.py?

    – iruvar
    Aug 14 '14 at 20:23












  • There are over 4000 genomes, each containing 1 or more contigs. would have to use my command line 4000+ times just to get the files i need.

    – Jeff
    Aug 14 '14 at 20:38






  • 1





    You use both folder and directory in your question. Is there a difference between those two things? If so, what is the difference? Also, please edit your question to include a portion of an example directory structure for us to look at.

    – dg99
    Aug 14 '14 at 20:46












  • A tree representing the directory hierarchy would be nice.

    – Cristian Ciupitu
    Aug 14 '14 at 20:56











  • @Jeff does script.py takes file names or directories? Do .fna and .faa come in pairs?

    – Simply_Me
    Aug 14 '14 at 21:48







3




3





Did you code script.py? Why not use os.walk to navigate the directory from within script.py?

– iruvar
Aug 14 '14 at 20:23






Did you code script.py? Why not use os.walk to navigate the directory from within script.py?

– iruvar
Aug 14 '14 at 20:23














There are over 4000 genomes, each containing 1 or more contigs. would have to use my command line 4000+ times just to get the files i need.

– Jeff
Aug 14 '14 at 20:38





There are over 4000 genomes, each containing 1 or more contigs. would have to use my command line 4000+ times just to get the files i need.

– Jeff
Aug 14 '14 at 20:38




1




1





You use both folder and directory in your question. Is there a difference between those two things? If so, what is the difference? Also, please edit your question to include a portion of an example directory structure for us to look at.

– dg99
Aug 14 '14 at 20:46






You use both folder and directory in your question. Is there a difference between those two things? If so, what is the difference? Also, please edit your question to include a portion of an example directory structure for us to look at.

– dg99
Aug 14 '14 at 20:46














A tree representing the directory hierarchy would be nice.

– Cristian Ciupitu
Aug 14 '14 at 20:56





A tree representing the directory hierarchy would be nice.

– Cristian Ciupitu
Aug 14 '14 at 20:56













@Jeff does script.py takes file names or directories? Do .fna and .faa come in pairs?

– Simply_Me
Aug 14 '14 at 21:48





@Jeff does script.py takes file names or directories? Do .fna and .faa come in pairs?

– Simply_Me
Aug 14 '14 at 21:48










1 Answer
1






active

oldest

votes


















0














I think this should help:



The script will use os.walk, as @1_CR suggested, and will call hello_world.py script with the path to both files when fna and faa files exist together (i.e. in same direcotry).



import glob, os

def scanfolder():
for path, dirs, files in os.walk('/home/shadowe/test1/test1/'):
flag_faa = 0
flag_fna = 0
for f in files:
if f.endswith('.faa'):
flag_faa = 1
faa_file_path = os.path.join(path,f)
if f.endswith('.fna'):
flag_fna = 1
fna_file_path = os.path.join(path,f)
if flag_faa == 1 and flag_fna == 1:
print "Calling script"
os.system("python hello_world.py" + " -a "+ faa_file_path + " -b " + fna_file_path)
flag_faa = 0
flag_fna = 0


scanfolder()


Second script:



import sys

print "Hello World"
print "This argument passing qualifies " + sys.argv[1] + " " + sys.argv[2] + " " + sys.argv[3] + " " + sys.argv[4]


Output:



$ python test.py 
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test2/test3/two.faa -b /home/shadowe/test1/test1/test2/test3/two.fna
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test6/test7/four.faa -b /home/shadowe/test1/test1/test6/test7/five.fna





share|improve this answer

























  • No such file or directory: 'CP003033.faa', is there a way to specify the path?

    – Jeff
    Aug 14 '14 at 21:46












  • @Jeff is that from your script.py?

    – Simply_Me
    Aug 14 '14 at 21:47











  • @Jeff do you need to pass file names as well?

    – Simply_Me
    Aug 14 '14 at 21:47











  • Yes I need to pass in 2 file names.

    – Jeff
    Aug 14 '14 at 21:48











  • For example: python myscript.py -a .faa -b .fna

    – Jeff
    Aug 14 '14 at 21:49











Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f150300%2frunning-python-script-over-multiple-directories-and-multiple-files%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









0














I think this should help:



The script will use os.walk, as @1_CR suggested, and will call hello_world.py script with the path to both files when fna and faa files exist together (i.e. in same direcotry).



import glob, os

def scanfolder():
for path, dirs, files in os.walk('/home/shadowe/test1/test1/'):
flag_faa = 0
flag_fna = 0
for f in files:
if f.endswith('.faa'):
flag_faa = 1
faa_file_path = os.path.join(path,f)
if f.endswith('.fna'):
flag_fna = 1
fna_file_path = os.path.join(path,f)
if flag_faa == 1 and flag_fna == 1:
print "Calling script"
os.system("python hello_world.py" + " -a "+ faa_file_path + " -b " + fna_file_path)
flag_faa = 0
flag_fna = 0


scanfolder()


Second script:



import sys

print "Hello World"
print "This argument passing qualifies " + sys.argv[1] + " " + sys.argv[2] + " " + sys.argv[3] + " " + sys.argv[4]


Output:



$ python test.py 
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test2/test3/two.faa -b /home/shadowe/test1/test1/test2/test3/two.fna
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test6/test7/four.faa -b /home/shadowe/test1/test1/test6/test7/five.fna





share|improve this answer

























  • No such file or directory: 'CP003033.faa', is there a way to specify the path?

    – Jeff
    Aug 14 '14 at 21:46












  • @Jeff is that from your script.py?

    – Simply_Me
    Aug 14 '14 at 21:47











  • @Jeff do you need to pass file names as well?

    – Simply_Me
    Aug 14 '14 at 21:47











  • Yes I need to pass in 2 file names.

    – Jeff
    Aug 14 '14 at 21:48











  • For example: python myscript.py -a .faa -b .fna

    – Jeff
    Aug 14 '14 at 21:49
















0














I think this should help:



The script will use os.walk, as @1_CR suggested, and will call hello_world.py script with the path to both files when fna and faa files exist together (i.e. in same direcotry).



import glob, os

def scanfolder():
for path, dirs, files in os.walk('/home/shadowe/test1/test1/'):
flag_faa = 0
flag_fna = 0
for f in files:
if f.endswith('.faa'):
flag_faa = 1
faa_file_path = os.path.join(path,f)
if f.endswith('.fna'):
flag_fna = 1
fna_file_path = os.path.join(path,f)
if flag_faa == 1 and flag_fna == 1:
print "Calling script"
os.system("python hello_world.py" + " -a "+ faa_file_path + " -b " + fna_file_path)
flag_faa = 0
flag_fna = 0


scanfolder()


Second script:



import sys

print "Hello World"
print "This argument passing qualifies " + sys.argv[1] + " " + sys.argv[2] + " " + sys.argv[3] + " " + sys.argv[4]


Output:



$ python test.py 
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test2/test3/two.faa -b /home/shadowe/test1/test1/test2/test3/two.fna
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test6/test7/four.faa -b /home/shadowe/test1/test1/test6/test7/five.fna





share|improve this answer

























  • No such file or directory: 'CP003033.faa', is there a way to specify the path?

    – Jeff
    Aug 14 '14 at 21:46












  • @Jeff is that from your script.py?

    – Simply_Me
    Aug 14 '14 at 21:47











  • @Jeff do you need to pass file names as well?

    – Simply_Me
    Aug 14 '14 at 21:47











  • Yes I need to pass in 2 file names.

    – Jeff
    Aug 14 '14 at 21:48











  • For example: python myscript.py -a .faa -b .fna

    – Jeff
    Aug 14 '14 at 21:49














0












0








0







I think this should help:



The script will use os.walk, as @1_CR suggested, and will call hello_world.py script with the path to both files when fna and faa files exist together (i.e. in same direcotry).



import glob, os

def scanfolder():
for path, dirs, files in os.walk('/home/shadowe/test1/test1/'):
flag_faa = 0
flag_fna = 0
for f in files:
if f.endswith('.faa'):
flag_faa = 1
faa_file_path = os.path.join(path,f)
if f.endswith('.fna'):
flag_fna = 1
fna_file_path = os.path.join(path,f)
if flag_faa == 1 and flag_fna == 1:
print "Calling script"
os.system("python hello_world.py" + " -a "+ faa_file_path + " -b " + fna_file_path)
flag_faa = 0
flag_fna = 0


scanfolder()


Second script:



import sys

print "Hello World"
print "This argument passing qualifies " + sys.argv[1] + " " + sys.argv[2] + " " + sys.argv[3] + " " + sys.argv[4]


Output:



$ python test.py 
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test2/test3/two.faa -b /home/shadowe/test1/test1/test2/test3/two.fna
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test6/test7/four.faa -b /home/shadowe/test1/test1/test6/test7/five.fna





share|improve this answer















I think this should help:



The script will use os.walk, as @1_CR suggested, and will call hello_world.py script with the path to both files when fna and faa files exist together (i.e. in same direcotry).



import glob, os

def scanfolder():
for path, dirs, files in os.walk('/home/shadowe/test1/test1/'):
flag_faa = 0
flag_fna = 0
for f in files:
if f.endswith('.faa'):
flag_faa = 1
faa_file_path = os.path.join(path,f)
if f.endswith('.fna'):
flag_fna = 1
fna_file_path = os.path.join(path,f)
if flag_faa == 1 and flag_fna == 1:
print "Calling script"
os.system("python hello_world.py" + " -a "+ faa_file_path + " -b " + fna_file_path)
flag_faa = 0
flag_fna = 0


scanfolder()


Second script:



import sys

print "Hello World"
print "This argument passing qualifies " + sys.argv[1] + " " + sys.argv[2] + " " + sys.argv[3] + " " + sys.argv[4]


Output:



$ python test.py 
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test2/test3/two.faa -b /home/shadowe/test1/test1/test2/test3/two.fna
Calling script
Hello World
This argument passing qualifies -a /home/shadowe/test1/test1/test6/test7/four.faa -b /home/shadowe/test1/test1/test6/test7/five.fna






share|improve this answer














share|improve this answer



share|improve this answer








edited Aug 14 '14 at 22:08

























answered Aug 14 '14 at 20:54









Simply_MeSimply_Me

1,4781810




1,4781810












  • No such file or directory: 'CP003033.faa', is there a way to specify the path?

    – Jeff
    Aug 14 '14 at 21:46












  • @Jeff is that from your script.py?

    – Simply_Me
    Aug 14 '14 at 21:47











  • @Jeff do you need to pass file names as well?

    – Simply_Me
    Aug 14 '14 at 21:47











  • Yes I need to pass in 2 file names.

    – Jeff
    Aug 14 '14 at 21:48











  • For example: python myscript.py -a .faa -b .fna

    – Jeff
    Aug 14 '14 at 21:49


















  • No such file or directory: 'CP003033.faa', is there a way to specify the path?

    – Jeff
    Aug 14 '14 at 21:46












  • @Jeff is that from your script.py?

    – Simply_Me
    Aug 14 '14 at 21:47











  • @Jeff do you need to pass file names as well?

    – Simply_Me
    Aug 14 '14 at 21:47











  • Yes I need to pass in 2 file names.

    – Jeff
    Aug 14 '14 at 21:48











  • For example: python myscript.py -a .faa -b .fna

    – Jeff
    Aug 14 '14 at 21:49

















No such file or directory: 'CP003033.faa', is there a way to specify the path?

– Jeff
Aug 14 '14 at 21:46






No such file or directory: 'CP003033.faa', is there a way to specify the path?

– Jeff
Aug 14 '14 at 21:46














@Jeff is that from your script.py?

– Simply_Me
Aug 14 '14 at 21:47





@Jeff is that from your script.py?

– Simply_Me
Aug 14 '14 at 21:47













@Jeff do you need to pass file names as well?

– Simply_Me
Aug 14 '14 at 21:47





@Jeff do you need to pass file names as well?

– Simply_Me
Aug 14 '14 at 21:47













Yes I need to pass in 2 file names.

– Jeff
Aug 14 '14 at 21:48





Yes I need to pass in 2 file names.

– Jeff
Aug 14 '14 at 21:48













For example: python myscript.py -a .faa -b .fna

– Jeff
Aug 14 '14 at 21:49






For example: python myscript.py -a .faa -b .fna

– Jeff
Aug 14 '14 at 21:49


















draft saved

draft discarded
















































Thanks for contributing an answer to Unix & Linux Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f150300%2frunning-python-script-over-multiple-directories-and-multiple-files%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown






Popular posts from this blog

Peggy Mitchell

Palaiologos

The Forum (Inglewood, California)