function readOnly(count){ }
Sign Up ›
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
Learn More

Browse by Topic
ShowAll Questionssorted byDate Posted
+ Start a Discussion
dmaysdmays 

How to increase batch size for insert

We're trying to do an initial load of Salesforce with Accounts and Contacts, but have run into a maximum batch size of 200 records.  Is there any way to increase this?

 

benjasikbenjasik
200 is the limit today.
dmaysdmays

Are you saying it cannot be increased?  We have 140,000 accounts to upload and our product people were told we would be able to do it in batches of 50,000.
SuperfellSuperfell
The limit is 200 in a single call, but you can call create as many times as you want.
benjasikbenjasik
Depending on your network connection, even using a batch of 200, we've seen load times of 20,000 to 120,000. It depends on how wide the records are, how much data you are loading, your network connection speed.

I don't think loading this many should take too long.

A batch of 50,000 would create a soap message that would be enormous and likely crash whatever soap client you are using due to memory issues.

Are you writing your own software to load the data, or using a third-party tool?

In 4.0, the limit was 2,000. In 5.0, when we added things like long text custom fields, we decreased the limit to 200 since the messages can be much larger. I think you'll still find the throughput to be good, even though you need to make more web services calls. The time to load into the system is larger than the network time.
dmaysdmays
Are there any C# code samples that demonstrate this?
darozdaroz

I'll give you some pseudo code if you like... What I did when I wrote my import util was to declare an sObject array (insSobjects[]). I also had a function that would 'append' another sObject to that array and resize the array. (In essence get length of array, make new one length+1, call Array.Copy, and add the new element on the end)

In my code I'd do something like this:

while (1==1) // Loop utnil whenever you're done...

{

        Account impAccount = new Account();

        // Do stuff here to popluate the impAccount object

        insSObjects = utils.addSobject(insSObjects, impAccount); // Add the new account to the sObject Array...

        if (insSObjects.Length >= 199)

        {

                // Call sfBinding.Create and process the results for errors

                insSObjects = new sObject[0];

        }

}

This would loop through, add my accounts, and send them to Salesforce in batches. I'll also note I had ALOT fewer records you do and I took the time in the weeks preceeding cutover to do alot of import runs and gave exception reports to my client to have the records 'fixed' before import day.

Best o' luck to ya.
dmaysdmays
Thanks.  That was very helpful.