Submit job only after previous job completes?
Clash Royale CLAN TAG#URR8PPP
up vote
-1
down vote
favorite
I have tried with this one:
cat sub-1.sh
#!/bin/sh
jid1=$(sbatch job1.sh)
#job can depend on a single job
jid2=$(sbatch --dependency=afterany:$jid1 job2.sh)
But it does not hold the job and showing this error:
sbatch: error: Batch job submission failed: Job dependency problem.
shell-script job-control batch-jobs
asked Sep 25 at 14:54
Saikat Pal
61
 |Â
show 1 more comment
up vote
-1
down vote
favorite
I have tried with this one:
cat sub-1.sh
#!/bin/sh
jid1=$(sbatch job1.sh)
#job can depend on a single job
jid2=$(sbatch --dependency=afterany:$jid1 job2.sh)
But it does not hold the job and showing this error:
sbatch: error: Batch job submission failed: Job dependency problem.
shell-script job-control batch-jobs
asked Sep 25 at 14:54
Saikat Pal
61
1
What do you mean "it fails"? Fails how? My guess would be that sbatch submits to the queue and then returns, so "it fails" means it doesn't wait.
– KevinO
Sep 25 at 15:10
Yes, Sir, your guess is right. All the jobs run simultaneously.
– Saikat Pal
Sep 25 at 15:34
In general, the approach would be that you get back the job id from the submission, then create a dependency on the next submission. See, e.g., this link. If what you are using doesn't support dependency chaining, then you have to do something to wait on notification. You can add to the submitted job an e-mail, or a posting to a queue, or you can poll the queue state using the returned jobid.
– KevinO
Sep 25 at 15:39
If you want the jobs to execute sequentially, why do you queue them?
– RalfFriedl
Sep 25 at 17:18
To continue on @RalfFriedl's train of thought; why do you queue them separately?
– Kusalananda
Sep 26 at 6:32
 |Â
show 1 more comment
up vote
-1
down vote
favorite
up vote
-1
down vote
favorite
I have tried with this one:
cat sub-1.sh
#!/bin/sh
jid1=$(sbatch job1.sh)
#job can depend on a single job
jid2=$(sbatch --dependency=afterany:$jid1 job2.sh)
But it does not hold the job and showing this error:
sbatch: error: Batch job submission failed: Job dependency problem.
shell-script job-control batch-jobs
asked Sep 25 at 14:54
Saikat Pal
61
I have tried with this one:
cat sub-1.sh
#!/bin/sh
jid1=$(sbatch job1.sh)
#job can depend on a single job
jid2=$(sbatch --dependency=afterany:$jid1 job2.sh)
But it does not hold the job and showing this error:
sbatch: error: Batch job submission failed: Job dependency problem.
shell-script job-control batch-jobs
shell-script job-control batch-jobs
asked Sep 25 at 14:54
Saikat Pal
61
asked Sep 25 at 14:54
Saikat Pal
61
edited Sep 26 at 5:54
asked Sep 25 at 14:54
Saikat Pal
61
asked Sep 25 at 14:54
Saikat Pal
61
asked Sep 25 at 14:54
Saikat Pal
61
61
1
What do you mean "it fails"? Fails how? My guess would be that sbatch submits to the queue and then returns, so "it fails" means it doesn't wait.
– KevinO
Sep 25 at 15:10
Yes, Sir, your guess is right. All the jobs run simultaneously.
– Saikat Pal
Sep 25 at 15:34
In general, the approach would be that you get back the job id from the submission, then create a dependency on the next submission. See, e.g., this link. If what you are using doesn't support dependency chaining, then you have to do something to wait on notification. You can add to the submitted job an e-mail, or a posting to a queue, or you can poll the queue state using the returned jobid.
– KevinO
Sep 25 at 15:39
If you want the jobs to execute sequentially, why do you queue them?
– RalfFriedl
Sep 25 at 17:18
To continue on @RalfFriedl's train of thought; why do you queue them separately?
– Kusalananda
Sep 26 at 6:32
 |Â
show 1 more comment
1
What do you mean "it fails"? Fails how? My guess would be that sbatch submits to the queue and then returns, so "it fails" means it doesn't wait.
– KevinO
Sep 25 at 15:10
Yes, Sir, your guess is right. All the jobs run simultaneously.
– Saikat Pal
Sep 25 at 15:34
In general, the approach would be that you get back the job id from the submission, then create a dependency on the next submission. See, e.g., this link. If what you are using doesn't support dependency chaining, then you have to do something to wait on notification. You can add to the submitted job an e-mail, or a posting to a queue, or you can poll the queue state using the returned jobid.
– KevinO
Sep 25 at 15:39
If you want the jobs to execute sequentially, why do you queue them?
– RalfFriedl
Sep 25 at 17:18
To continue on @RalfFriedl's train of thought; why do you queue them separately?
– Kusalananda
Sep 26 at 6:32
1
1
What do you mean "it fails"? Fails how? My guess would be that sbatch submits to the queue and then returns, so "it fails" means it doesn't wait.
– KevinO
Sep 25 at 15:10
What do you mean "it fails"? Fails how? My guess would be that sbatch submits to the queue and then returns, so "it fails" means it doesn't wait.
– KevinO
Sep 25 at 15:10
Yes, Sir, your guess is right. All the jobs run simultaneously.
– Saikat Pal
Sep 25 at 15:34
Yes, Sir, your guess is right. All the jobs run simultaneously.
– Saikat Pal
Sep 25 at 15:34
In general, the approach would be that you get back the job id from the submission, then create a dependency on the next submission. See, e.g., this link. If what you are using doesn't support dependency chaining, then you have to do something to wait on notification. You can add to the submitted job an e-mail, or a posting to a queue, or you can poll the queue state using the returned jobid.
– KevinO
Sep 25 at 15:39
In general, the approach would be that you get back the job id from the submission, then create a dependency on the next submission. See, e.g., this link. If what you are using doesn't support dependency chaining, then you have to do something to wait on notification. You can add to the submitted job an e-mail, or a posting to a queue, or you can poll the queue state using the returned jobid.
– KevinO
Sep 25 at 15:39
If you want the jobs to execute sequentially, why do you queue them?
– RalfFriedl
Sep 25 at 17:18
If you want the jobs to execute sequentially, why do you queue them?
– RalfFriedl
Sep 25 at 17:18
To continue on @RalfFriedl's train of thought; why do you queue them separately?
– Kusalananda
Sep 26 at 6:32
To continue on @RalfFriedl's train of thought; why do you queue them separately?
– Kusalananda
Sep 26 at 6:32
 |Â
show 1 more comment
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Â
draft saved
draft discarded
Â
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f471351%2fsubmit-job-only-after-previous-job-completes%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
What do you mean "it fails"? Fails how? My guess would be that sbatch submits to the queue and then returns, so "it fails" means it doesn't wait.
– KevinO
Sep 25 at 15:10
Yes, Sir, your guess is right. All the jobs run simultaneously.
– Saikat Pal
Sep 25 at 15:34
In general, the approach would be that you get back the job id from the submission, then create a dependency on the next submission. See, e.g., this link. If what you are using doesn't support dependency chaining, then you have to do something to wait on notification. You can add to the submitted job an e-mail, or a posting to a queue, or you can poll the queue state using the returned jobid.
– KevinO
Sep 25 at 15:39
If you want the jobs to execute sequentially, why do you queue them?
– RalfFriedl
Sep 25 at 17:18
To continue on @RalfFriedl's train of thought; why do you queue them separately?
– Kusalananda
Sep 26 at 6:32