restore $0 or $BASH_SOURCE after it is modified by the cluster
Clash Royale CLAN TAG#URR8PPP
I am using a shared SLURM cluster. I am trying to get the path of the bash script from inside the script itself.
There is already an excellent thread here: https://stackoverflow.com/questions/59895/get-the-source-directory-of-a-bash-script-from-within-the-script-itself. Unfortunately, none of those solutions work for me. They work when I execute the bash script normally, but not when I submit the script as a job. I tried $BASH_SOURCE, $0, $_, and readlink /proc/$$/fd/255. They all return something like /cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script, which is a path auto-generated by SLURM and has nothing to do with the script I submitted. I doubt there are any other solutions that directly answer my question that are not already mentioned, but is there maybe some way to override whatever the cluster is doing? Can I somehow restore standard bash behavior?
bash shell-script cluster slurm
asked Feb 5 at 16:21
burgerburger
1375
|
show 6 more comments
I am using a shared SLURM cluster. I am trying to get the path of the bash script from inside the script itself.
There is already an excellent thread here: https://stackoverflow.com/questions/59895/get-the-source-directory-of-a-bash-script-from-within-the-script-itself. Unfortunately, none of those solutions work for me. They work when I execute the bash script normally, but not when I submit the script as a job. I tried $BASH_SOURCE, $0, $_, and readlink /proc/$$/fd/255. They all return something like /cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script, which is a path auto-generated by SLURM and has nothing to do with the script I submitted. I doubt there are any other solutions that directly answer my question that are not already mentioned, but is there maybe some way to override whatever the cluster is doing? Can I somehow restore standard bash behavior?
bash shell-script cluster slurm
asked Feb 5 at 16:21
burgerburger
1375
1
It seems to me like you have recovered the name of the script:/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script. What were you expecting instead, and why does it have to be different?
– Jeff Schaller
Feb 5 at 16:30
1
Why do you think that is not the path of the script that's executing?
– Jeff Schaller
Feb 5 at 16:33
1
I don't use slurm, but it's conceivable to me that the system would copy the source script to a worker node in some sort of spooling directory for that node to execute it. Sounds like you might have different expectations?
– Jeff Schaller
Feb 5 at 16:37
2
SLURM copies your script. On a SLURM cluster, you may not even have a home directory, so the working directory may well be at another path than what you would see when you run your script locally.
– Kusalananda
Feb 5 at 16:38
1
I've gone in a circle now, so I may have to give up, but -- you start off knowing the path of your script and then you find the new location when run by slurm, but now you want to know the original location?
– Jeff Schaller
Feb 5 at 16:47
|
show 6 more comments
I am using a shared SLURM cluster. I am trying to get the path of the bash script from inside the script itself.
There is already an excellent thread here: https://stackoverflow.com/questions/59895/get-the-source-directory-of-a-bash-script-from-within-the-script-itself. Unfortunately, none of those solutions work for me. They work when I execute the bash script normally, but not when I submit the script as a job. I tried $BASH_SOURCE, $0, $_, and readlink /proc/$$/fd/255. They all return something like /cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script, which is a path auto-generated by SLURM and has nothing to do with the script I submitted. I doubt there are any other solutions that directly answer my question that are not already mentioned, but is there maybe some way to override whatever the cluster is doing? Can I somehow restore standard bash behavior?
bash shell-script cluster slurm
asked Feb 5 at 16:21
burgerburger
1375
I am using a shared SLURM cluster. I am trying to get the path of the bash script from inside the script itself.
There is already an excellent thread here: https://stackoverflow.com/questions/59895/get-the-source-directory-of-a-bash-script-from-within-the-script-itself. Unfortunately, none of those solutions work for me. They work when I execute the bash script normally, but not when I submit the script as a job. I tried $BASH_SOURCE, $0, $_, and readlink /proc/$$/fd/255. They all return something like /cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script, which is a path auto-generated by SLURM and has nothing to do with the script I submitted. I doubt there are any other solutions that directly answer my question that are not already mentioned, but is there maybe some way to override whatever the cluster is doing? Can I somehow restore standard bash behavior?
bash shell-script cluster slurm
bash shell-script cluster slurm
asked Feb 5 at 16:21
burgerburger
1375
asked Feb 5 at 16:21
burgerburger
1375
edited Feb 5 at 16:34
burger
asked Feb 5 at 16:21
burgerburger
1375
asked Feb 5 at 16:21
burgerburger
1375
asked Feb 5 at 16:21
burgerburger
1375
1375
1
It seems to me like you have recovered the name of the script:/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script. What were you expecting instead, and why does it have to be different?
– Jeff Schaller
Feb 5 at 16:30
1
Why do you think that is not the path of the script that's executing?
– Jeff Schaller
Feb 5 at 16:33
1
I don't use slurm, but it's conceivable to me that the system would copy the source script to a worker node in some sort of spooling directory for that node to execute it. Sounds like you might have different expectations?
– Jeff Schaller
Feb 5 at 16:37
2
SLURM copies your script. On a SLURM cluster, you may not even have a home directory, so the working directory may well be at another path than what you would see when you run your script locally.
– Kusalananda
Feb 5 at 16:38
1
I've gone in a circle now, so I may have to give up, but -- you start off knowing the path of your script and then you find the new location when run by slurm, but now you want to know the original location?
– Jeff Schaller
Feb 5 at 16:47
|
show 6 more comments
1
It seems to me like you have recovered the name of the script:/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script. What were you expecting instead, and why does it have to be different?
– Jeff Schaller
Feb 5 at 16:30
1
Why do you think that is not the path of the script that's executing?
– Jeff Schaller
Feb 5 at 16:33
1
I don't use slurm, but it's conceivable to me that the system would copy the source script to a worker node in some sort of spooling directory for that node to execute it. Sounds like you might have different expectations?
– Jeff Schaller
Feb 5 at 16:37
2
SLURM copies your script. On a SLURM cluster, you may not even have a home directory, so the working directory may well be at another path than what you would see when you run your script locally.
– Kusalananda
Feb 5 at 16:38
1
I've gone in a circle now, so I may have to give up, but -- you start off knowing the path of your script and then you find the new location when run by slurm, but now you want to know the original location?
– Jeff Schaller
Feb 5 at 16:47
1
1
It seems to me like you have recovered the name of the script:
/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script. What were you expecting instead, and why does it have to be different?– Jeff Schaller
Feb 5 at 16:30
It seems to me like you have recovered the name of the script:
/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script. What were you expecting instead, and why does it have to be different?– Jeff Schaller
Feb 5 at 16:30
1
1
Why do you think that is not the path of the script that's executing?
– Jeff Schaller
Feb 5 at 16:33
Why do you think that is not the path of the script that's executing?
– Jeff Schaller
Feb 5 at 16:33
1
1
I don't use slurm, but it's conceivable to me that the system would copy the source script to a worker node in some sort of spooling directory for that node to execute it. Sounds like you might have different expectations?
– Jeff Schaller
Feb 5 at 16:37
I don't use slurm, but it's conceivable to me that the system would copy the source script to a worker node in some sort of spooling directory for that node to execute it. Sounds like you might have different expectations?
– Jeff Schaller
Feb 5 at 16:37
2
2
SLURM copies your script. On a SLURM cluster, you may not even have a home directory, so the working directory may well be at another path than what you would see when you run your script locally.
– Kusalananda
Feb 5 at 16:38
SLURM copies your script. On a SLURM cluster, you may not even have a home directory, so the working directory may well be at another path than what you would see when you run your script locally.
– Kusalananda
Feb 5 at 16:38
1
1
I've gone in a circle now, so I may have to give up, but -- you start off knowing the path of your script and then you find the new location when run by slurm, but now you want to know the original location?
– Jeff Schaller
Feb 5 at 16:47
I've gone in a circle now, so I may have to give up, but -- you start off knowing the path of your script and then you find the new location when run by slurm, but now you want to know the original location?
– Jeff Schaller
Feb 5 at 16:47
|
show 6 more comments
1 Answer
1
active
oldest
votes
The problem was resolved in the comments.
To summarize: It turns out that I did not properly diagnose the initial problem. SLURM did not modify $BASH_SOURCE or $0. I assumed it simply executed my script, but it actually copied my script to a new location (/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script). To get the behavior I expected, I submitted the job with the --wrap parameter which wraps the specified command string in a shell script.
answered Feb 22 at 15:55
burgerburger
1375
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f498844%2frestore-0-or-bash-source-after-it-is-modified-by-the-cluster%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The problem was resolved in the comments.
To summarize: It turns out that I did not properly diagnose the initial problem. SLURM did not modify $BASH_SOURCE or $0. I assumed it simply executed my script, but it actually copied my script to a new location (/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script). To get the behavior I expected, I submitted the job with the --wrap parameter which wraps the specified command string in a shell script.
answered Feb 22 at 15:55
burgerburger
1375
add a comment |
The problem was resolved in the comments.
To summarize: It turns out that I did not properly diagnose the initial problem. SLURM did not modify $BASH_SOURCE or $0. I assumed it simply executed my script, but it actually copied my script to a new location (/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script). To get the behavior I expected, I submitted the job with the --wrap parameter which wraps the specified command string in a shell script.
answered Feb 22 at 15:55
burgerburger
1375
add a comment |
The problem was resolved in the comments.
To summarize: It turns out that I did not properly diagnose the initial problem. SLURM did not modify $BASH_SOURCE or $0. I assumed it simply executed my script, but it actually copied my script to a new location (/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script). To get the behavior I expected, I submitted the job with the --wrap parameter which wraps the specified command string in a shell script.
answered Feb 22 at 15:55
burgerburger
1375
The problem was resolved in the comments.
To summarize: It turns out that I did not properly diagnose the initial problem. SLURM did not modify $BASH_SOURCE or $0. I assumed it simply executed my script, but it actually copied my script to a new location (/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script). To get the behavior I expected, I submitted the job with the --wrap parameter which wraps the specified command string in a shell script.
answered Feb 22 at 15:55
burgerburger
1375
answered Feb 22 at 15:55
burgerburger
1375
answered Feb 22 at 15:55
burgerburger
1375
answered Feb 22 at 15:55
burgerburger
1375
1375
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f498844%2frestore-0-or-bash-source-after-it-is-modified-by-the-cluster%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
It seems to me like you have recovered the name of the script:
/cm/local/apps/slurm/var/spool/jobXXXXXX/slurm_script. What were you expecting instead, and why does it have to be different?– Jeff Schaller
Feb 5 at 16:30
1
Why do you think that is not the path of the script that's executing?
– Jeff Schaller
Feb 5 at 16:33
1
I don't use slurm, but it's conceivable to me that the system would copy the source script to a worker node in some sort of spooling directory for that node to execute it. Sounds like you might have different expectations?
– Jeff Schaller
Feb 5 at 16:37
2
SLURM copies your script. On a SLURM cluster, you may not even have a home directory, so the working directory may well be at another path than what you would see when you run your script locally.
– Kusalananda
Feb 5 at 16:38
1
I've gone in a circle now, so I may have to give up, but -- you start off knowing the path of your script and then you find the new location when run by slurm, but now you want to know the original location?
– Jeff Schaller
Feb 5 at 16:47