Enforce Batches (whole run, not single jobs) to wait for each other
Clash Royale CLAN TAG#URR8PPP
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
|
show 1 more comment
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
Dec 12 at 12:00
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
Dec 12 at 12:07
4
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
Dec 12 at 12:11
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
Dec 12 at 12:16
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
Dec 12 at 12:35
|
show 1 more comment
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
A few situations in my app trigger a Clean up batch to run. It will process many records. In some situations multiple of such Cleanup batches will be triggered by independent user operations.
If the batch work on the same record we sometime see locking or currupt data.
Is there any means to sequence batches of the same type (class) without complicated scheduling solutions? Maybe with the FlexQueue and its Holding state?
apex batch asynchronous apex-flex-queue
apex batch asynchronous apex-flex-queue
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
edited Dec 12 at 13:42
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
asked Dec 12 at 11:56
Robert Sösemann
12.6k1074209
12.6k1074209
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
Dec 12 at 12:00
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
Dec 12 at 12:07
4
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
Dec 12 at 12:11
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
Dec 12 at 12:16
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
Dec 12 at 12:35
|
show 1 more comment
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
Dec 12 at 12:00
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
Dec 12 at 12:07
4
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
Dec 12 at 12:11
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
Dec 12 at 12:16
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
Dec 12 at 12:35
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
Dec 12 at 12:00
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
Dec 12 at 12:00
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
Dec 12 at 12:07
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
Dec 12 at 12:07
4
4
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
Dec 12 at 12:11
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
Dec 12 at 12:11
1
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
Dec 12 at 12:16
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
Dec 12 at 12:16
1
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
Dec 12 at 12:35
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
Dec 12 at 12:35
|
show 1 more comment
2 Answers
2
active
oldest
votes
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
answered Dec 12 at 12:44
sfdcfox
246k11186423
add a comment |
If the batch query is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we don't need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId)
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty())
for(ApexAsyncJob as : asyncJobList) as.Status == 'Preparing') //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
else
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
if(isNeedToInitiateNewBatch)
//Initiate Batch
edited Dec 12 at 16:02
battery.cord
6,68051744
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "459"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f242299%2fenforce-batches-whole-run-not-single-jobs-to-wait-for-each-other%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
answered Dec 12 at 12:44
sfdcfox
246k11186423
add a comment |
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
answered Dec 12 at 12:44
sfdcfox
246k11186423
add a comment |
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
answered Dec 12 at 12:44
sfdcfox
246k11186423
Typical designs for this involve a temporary object. The idea is the code that kicks off the deletion inserts some sort of temporary record, and then checks if a batch is running or queued. If so, it skips calling the batch. The batch itself checks if there's any temporary records to process in its finish method, and if so, restarts the batch with a fresh query. This is similar to the Mutex idea from Aidan, but allows potential restarts of failed records; one temporary record per record to delete. Having this second object should mostly help resolve the problem, although specific versions of this might need additional work. It's kind of hard to predict ahead of time, as it depends on the types of relationships and objects involved. For example, you might want to upsert these temporary records somehow to avoid duplicates, which would slightly increase database contention but may reduce number of batches that need to run.
answered Dec 12 at 12:44
sfdcfox
246k11186423
answered Dec 12 at 12:44
sfdcfox
246k11186423
answered Dec 12 at 12:44
sfdcfox
246k11186423
answered Dec 12 at 12:44
sfdcfox
246k11186423
246k11186423
add a comment |
add a comment |
If the batch query is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we don't need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId)
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty())
for(ApexAsyncJob as : asyncJobList) as.Status == 'Preparing') //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
else
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
if(isNeedToInitiateNewBatch)
//Initiate Batch
edited Dec 12 at 16:02
battery.cord
6,68051744
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
add a comment |
If the batch query is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we don't need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId)
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty())
for(ApexAsyncJob as : asyncJobList) as.Status == 'Preparing') //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
else
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
if(isNeedToInitiateNewBatch)
//Initiate Batch
edited Dec 12 at 16:02
battery.cord
6,68051744
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
add a comment |
If the batch query is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we don't need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId)
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty())
for(ApexAsyncJob as : asyncJobList) as.Status == 'Preparing') //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
else
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
if(isNeedToInitiateNewBatch)
//Initiate Batch
edited Dec 12 at 16:02
battery.cord
6,68051744
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
If the batch query is same, then it makes sense to abort the currently running batch and execute a new one which will also pick up records of last batch.
Also, if the batch is in Holding or Queued state that means the Start method was not called and thus when start method will be called, it will also pick up new records, meaning we don't need to run a new batch.
public static void addBatchIfNotRunning(Id apexBatchClassId)
ApexAsyncJob asyncJobList = [Select Id From ApexAsyncJob where ApexClassId=:apexBatchClassId AND JobType='BatchApex' and Status In ('Processing','Preparing','Holding','Queued')];
boolean isNeedToInitiateNewBatch = true;
if(!asyncJobList.isEmpty())
for(ApexAsyncJob as : asyncJobList) as.Status == 'Preparing') //As start method of batch is running or run, it wont pick new records so abort it
System.abortJob(as.Id);
else
//Batch is already queued or in Holding stage so it will pick up new records
isNeedToInitiateNewBatch = false;
if(isNeedToInitiateNewBatch)
//Initiate Batch
edited Dec 12 at 16:02
battery.cord
6,68051744
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
edited Dec 12 at 16:02
battery.cord
6,68051744
edited Dec 12 at 16:02
battery.cord
6,68051744
edited Dec 12 at 16:02
battery.cord
6,68051744
6,68051744
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
answered Dec 12 at 13:02
Pranay Jaiswal
13.1k32351
13.1k32351
add a comment |
add a comment |
Thanks for contributing an answer to Salesforce Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsalesforce.stackexchange.com%2fquestions%2f242299%2fenforce-batches-whole-run-not-single-jobs-to-wait-for-each-other%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
We can run next batch from first batch finish method. So it will execute in sequence.
– SFDC Learner
Dec 12 at 12:00
Batch chaining only works when the first batch knows the second batch. This is not the case.
– Robert Sösemann
Dec 12 at 12:07
4
I don't think there is, but I have some code I can share with you that does the job. My solution uses a custom object, Mutex__c, to ensure that only one instance of a batch class is running at a time. Typically, that instance processes all applicable records, and keeps re-queueing itself until everything is processed. Any new attempts to start a batch go via the Mutex__c object, so they do nothing if it's already running. It even goes so far as abstracting away the async implementation so you can run your code as batch/queuable based on metadata settings
– Aidan
Dec 12 at 12:11
1
Is the batch Querry Same? if yes you can querry ApexAsyncJob to see if any batch is running or is in holding state. If running then initiate the new batch by aborting the old one. If there is no batch waiting then create a new batch as usual
– Pranay Jaiswal
Dec 12 at 12:16
1
@PranayJaiswal: Sound like a great idea: aborting the old one. You should add this as a solution with a code example. But for me this is not enough as the batch runs on different records. I might loose stuff.
– Robert Sösemann
Dec 12 at 12:35