openmpi and read file in lines
Clash Royale CLAN TAG#URR8PPP
I need to run several tasks in sequence. The task names are listed in a file (jobs.txt). I use the following code (example):
#! /bin/bash
while IFS=$'r' read -r line || [[ -n "$line" ]]; do
echo $line
done < "jobs.txt"
It works well. However, I need to run these tasks with openmpi (in this example, change "echo $line" to "mpirun -n 4 echo $line"), then the script does not work any more... It only read the first line from "jobs.txt" and stopped.
Any idea what's the problem and how to fix it?
linux bash
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
asked Feb 10 at 22:06
michael morganmichael morgan
255
add a comment |
I need to run several tasks in sequence. The task names are listed in a file (jobs.txt). I use the following code (example):
#! /bin/bash
while IFS=$'r' read -r line || [[ -n "$line" ]]; do
echo $line
done < "jobs.txt"
It works well. However, I need to run these tasks with openmpi (in this example, change "echo $line" to "mpirun -n 4 echo $line"), then the script does not work any more... It only read the first line from "jobs.txt" and stopped.
Any idea what's the problem and how to fix it?
linux bash
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
asked Feb 10 at 22:06
michael morganmichael morgan
255
I suspect it's because the firstmpirunprocess "eats" the standard input that you are trying to read in the loop - similar to what happens here: Problem using read command within while read loop. You may be able to prevent that by using a different file descriptor for the loop e.g.read -u3 -r line ... 3< jobs.txtbut I don't know enough about OpenMPI to say whether that's the right thing to do.
– steeldriver
Feb 10 at 22:29
add a comment |
I need to run several tasks in sequence. The task names are listed in a file (jobs.txt). I use the following code (example):
#! /bin/bash
while IFS=$'r' read -r line || [[ -n "$line" ]]; do
echo $line
done < "jobs.txt"
It works well. However, I need to run these tasks with openmpi (in this example, change "echo $line" to "mpirun -n 4 echo $line"), then the script does not work any more... It only read the first line from "jobs.txt" and stopped.
Any idea what's the problem and how to fix it?
linux bash
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
asked Feb 10 at 22:06
michael morganmichael morgan
255
I need to run several tasks in sequence. The task names are listed in a file (jobs.txt). I use the following code (example):
#! /bin/bash
while IFS=$'r' read -r line || [[ -n "$line" ]]; do
echo $line
done < "jobs.txt"
It works well. However, I need to run these tasks with openmpi (in this example, change "echo $line" to "mpirun -n 4 echo $line"), then the script does not work any more... It only read the first line from "jobs.txt" and stopped.
Any idea what's the problem and how to fix it?
linux bash
linux bash
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
asked Feb 10 at 22:06
michael morganmichael morgan
255
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
asked Feb 10 at 22:06
michael morganmichael morgan
255
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
edited Feb 11 at 6:30
Rui F Ribeiro
41.1k1479137
41.1k1479137
asked Feb 10 at 22:06
michael morganmichael morgan
255
asked Feb 10 at 22:06
michael morganmichael morgan
255
asked Feb 10 at 22:06
michael morganmichael morgan
255
255
I suspect it's because the firstmpirunprocess "eats" the standard input that you are trying to read in the loop - similar to what happens here: Problem using read command within while read loop. You may be able to prevent that by using a different file descriptor for the loop e.g.read -u3 -r line ... 3< jobs.txtbut I don't know enough about OpenMPI to say whether that's the right thing to do.
– steeldriver
Feb 10 at 22:29
add a comment |
I suspect it's because the firstmpirunprocess "eats" the standard input that you are trying to read in the loop - similar to what happens here: Problem using read command within while read loop. You may be able to prevent that by using a different file descriptor for the loop e.g.read -u3 -r line ... 3< jobs.txtbut I don't know enough about OpenMPI to say whether that's the right thing to do.
– steeldriver
Feb 10 at 22:29
I suspect it's because the first
mpirun process "eats" the standard input that you are trying to read in the loop - similar to what happens here: Problem using read command within while read loop. You may be able to prevent that by using a different file descriptor for the loop e.g. read -u3 -r line ... 3< jobs.txt but I don't know enough about OpenMPI to say whether that's the right thing to do.– steeldriver
Feb 10 at 22:29
I suspect it's because the first
mpirun process "eats" the standard input that you are trying to read in the loop - similar to what happens here: Problem using read command within while read loop. You may be able to prevent that by using a different file descriptor for the loop e.g. read -u3 -r line ... 3< jobs.txt but I don't know enough about OpenMPI to say whether that's the right thing to do.– steeldriver
Feb 10 at 22:29
add a comment |
1 Answer
1
active
oldest
votes
Yes by assigning file descriptor other than stdin problem solved.
answered Feb 12 at 22:50
michael morganmichael morgan
255
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f499828%2fopenmpi-and-read-file-in-lines%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Yes by assigning file descriptor other than stdin problem solved.
answered Feb 12 at 22:50
michael morganmichael morgan
255
add a comment |
Yes by assigning file descriptor other than stdin problem solved.
answered Feb 12 at 22:50
michael morganmichael morgan
255
add a comment |
Yes by assigning file descriptor other than stdin problem solved.
answered Feb 12 at 22:50
michael morganmichael morgan
255
Yes by assigning file descriptor other than stdin problem solved.
answered Feb 12 at 22:50
michael morganmichael morgan
255
answered Feb 12 at 22:50
michael morganmichael morgan
255
answered Feb 12 at 22:50
michael morganmichael morgan
255
answered Feb 12 at 22:50
michael morganmichael morgan
255
255
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f499828%2fopenmpi-and-read-file-in-lines%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I suspect it's because the first
mpirunprocess "eats" the standard input that you are trying to read in the loop - similar to what happens here: Problem using read command within while read loop. You may be able to prevent that by using a different file descriptor for the loop e.g.read -u3 -r line ... 3< jobs.txtbut I don't know enough about OpenMPI to say whether that's the right thing to do.– steeldriver
Feb 10 at 22:29