function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion

Getting the same 200 only records in each trigger when using BULK API via Data Loader

We have written a de-duplication logic for contact records where we call a batch job from trigger (Yes, it sounds weird but the only thing that seems to work as we have variable criteria for each account). To overcome batch schedule limit of 5, we are using data loader with bulk API enabled and the natch size set to 1000 so that we can upload 5000 records successfully without hitting the 5 batch job limit. When I am testing with 3000 thousand contact records, let say they are named from Test0001 to Test3000 I am observing a strange behavior.

For 3000 records, 3 batch jobs start to run (as batch size is 1000). I am passing the newly inserted records in parameters to the stateful batch class. What I expect is that 1000 records will be passed for each of the 3 batch jobs and they will be compared to existing records for duplicates (which I query in start method of batch) but I only get Test0001 to Test0200 i.e. from batch of 1000 records inserted via data loader API only FIRST 200 records are passed in parameter to the batch class and rest 800 are not. This is something strance as it means only first 200 records are processes if I insert using a batch size of 1000 through data loader with Bulk API enabled.

Has anyone of you encountered this issue or have any ideas to share on how to deal with it? I can share code as well but I think the question is more conceptual. Any help is much appreciated.


You have made batch size 1000 but it is my assumption that trigger will always process 200 records at one time. So when you insert 1000 records through bulk API using dataloader, 1000 records will be inserted but trigger will break it into 5 chunks with 200 records each. You will be passing newly inserted records from trigger using (which contains only 200 records) to batch apex. Thats why you are able to see only 200 records (Test0001 to Test0200). Other records will be passed in other transaction.

Hope this will help you.
[If it solves your problem, please mark it as solution]

Sunil Kumar
Well in that case the batch job called from trigger should be called 15 times as I am inserting 3000 records (200*15 for 3000 records) and I will hit the 5 batch job limit. But that's not happening, for 3000 records only 3 batch jobs are running as the batch size is 1000 in data loader. I just want these 1000 records to be passed as parameter to batch class but I am getting first 200. Where are rest 800 going? The next job gets records from Test1000 to Test1200.