function readOnly(count){ }
Sign Up ›
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
Learn More

Browse by Topic
ShowAll Questionssorted byDate Posted
+ Start a Discussion
Srinivasulu BodapatiSrinivasulu Bodapati 

Bulk API - Add a Batch to a Job

Hi Everyone,

We are integrating Oracle system with Salesforce. We are using a tool called Websphere Cast Iron to achieve the integration.

The tool provides us set of connectors to execute Salesforce Bulk APIs.

Problem:
-----------
Oracle source tables have huge data,we are fetching 10k records from source tables and submitting the batch to Salesforce and querying its status after couple of minutes. When we query for the batch results we are getting a valid response of either record created successfully or failed.

But the problem is the result row doesn't have "ExternalID" field defined in custom object, it has only 18 character "id", success and error elements. if we have "ExternalID" in result row along with"id" it would be easy for us to identify and update corresponding record in source table.

You may have suggestions like go and query salesforce with "id" and get ExternalID to update your source tables data.

However we want to know is there any work around to get "ExternalID" in batch results itself?

Second query is, Can we implement a logic to generate a unique 18 character sequence(Source table Primary key column + some character string) and set it as "id" while submitting the job itself, so when we get the batch result we know which part of the "id" is our primary key and update that record in source table with the "id".

Any suggestions please.

Thanks & Regards,
Srinivas
 
Austin MuellerAustin Mueller
Batch processing goes in chronological order for each record, so you might be able to do something like this:
  1. Create a database table in oracle with the following fields
    1. ExternalID
    2. BatchID
    3. RecordID
    4. Success
    5. Created
    6. Error
  2. Insert the records in the batch into the table (same order as they are sent to the Bulk API) along with the Batch ID
    1. ExternalID = 'abc'
    2. BatchID = '123456789012345678'
  3. When the batch is finished, get the results back, query your oracle table by the Batch ID.
    1. SELECT ExternalID FROM TableName WHERE BatchID = '123456789012345678'
  4. Loop through each result and the Salesforce ID should match up to the External ID in terms of the record, if it doesn't have a Salesforce ID, it will most likely have the error message for the ExternalID you tried to insert / update.

 
Srinivasulu BodapatiSrinivasulu Bodapati
Thank you Austin for your proposed solution and we accept that it will resolve the problem.

However, we felt loop through each result of 10k records may create a performance overhead in our tool.In a day we may need to migrate 300k records from oracle to salesforce.

If the salesforce batch results itself can return the External ID along with Salesforce ID then we can easily do DB batch update to map Salesforce ID with External ID record in source tables.