function readOnly(count){ }
Sign Up ›
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
Learn More

Browse by Topic
ShowAll Questionssorted byDate Posted
+ Start a Discussion
Kevin DavisKevin Davis 

PHP Upsert Limit

Hi there,

 

I'm seem to be having a problem with the PHP Upsert Limit.

 

Here is the situtation. I have a dataset with records over 200. Is there a way to override the limit?

 

Here is the code so far:

 

while ($record_count == $number_records)  

{

  set_time_limit(20);

 {$data_set = mysql_query("SELECT * from table_name") or die (mysql_error());}

 $data_count=mysql_num_rows($data_set);   

 

 $all_fields = array();
  $i = 0;

while ($data_1 = mysql_fetch_assoc($data_set))
  {
     $all_fields[$i]['RecordTypeId']                    = $data_1['field1'];
     $all_fields[$i]['STB_Organization_ID__c']          = $data_1['field2'];
     $all_fields[$i]['Branch_Site_ID__c']               = $data_1['field3'];
    
     $i++;
   }

 

 $record_count = $i;
  $sObjects = array();

   foreach ($all_fields as $fieldset)
   {
     $sObject = new sObject();
     $sObject->type = 'Account'; // Salesforce Table or object that you will perform the upsert on
     $sObject->fields = $fieldset;
     array_push($sObjects, $sObject);
   }
 
   
   $success = upsert_accounts($client, $sObjects, $file_updated, $file_created, $file_failed);

    if (is_array($success))
    {
    
      $accounts_created = $accounts_created + $success[0];
      $accounts_updated = $accounts_updated + $success[1];
      $accounts_failed =  $accounts_failed + $success[2];
      
    }

 

    $starting_record = $starting_record + $number_records;
    
    ob_start();    
    $total_record_count = $total_record_count + $record_count;
    echo $total_record_count." records processed.<br />";
    ob_end_flush();

 

 

}

Pat PattersonPat Patterson

This limit is fixed for the API - there is no override. You'll need to split your upsert calls into batches of 200 records.
Kevin DavisKevin Davis

Hi Pat,

 

So what I need to do is the following:

 

1. Read the entire dataset first.

2. Then break the dataset up by 200 records until it is compelted.

 

Is that correct? Is so do you have any suggestions?

 

Thank you..
Pat PattersonPat Patterson

Hi Kevin,

 

Exactly - you would refactor the code along these lines:

 

 

  $sObjects = array();
  $i = 0;

  foreach ($all_fields as $fieldset) {
    $sObject = new sObject();
    $sObject->type = 'Account'; // Salesforce Table or object that you will perform the upsert on
    $sObject->fields = $fieldset;
    array_push($sObjects, $sObject);
    $i++;

    if ($i == 200) {
      $success = upsert_accounts($client, $sObjects, $file_updated, $file_created, $file_failed);

       if (is_array($success)) {
         $accounts_created = $accounts_created + $success[0];
         $accounts_updated = $accounts_updated + $success[1];
         $accounts_failed =  $accounts_failed + $success[2];
       }
       $sObjects = array();
       $i = 0;
    }
  }

  
  $success = upsert_accounts($client, $sObjects, $file_updated, $file_created, $file_failed);

  if (is_array($success)) {
    $accounts_created = $accounts_created + $success[0];
    $accounts_updated = $accounts_updated + $success[1];
    $accounts_failed =  $accounts_failed + $success[2];
  }

 

NOTE - I haven't tested the above, but it looks like it would work. The refactored code simple calls upsert_accounts() for every 200 records (within the loop) then for the remainder (after the loop).

 

 

You would probably want to define a constant rather than use the literal value 200 in the code, then if things change anytime in the future it will be easy to update your code.
Kevin DavisKevin Davis

Hi there... Sorry for the delay in the response.. 

 

I tried that solution and it didn't work so I tried something else.

 

Here is what I did:

 

1. I set the number of records at 200

2. I had the record count = number of records

3. Starting record  = 1

 

In the while loop I did the following:

 

rec_pross =  starting_record + (number_records-1);

 

Then here is my sql statement:

 

SELECT * from tbl_name where tbl_id BETWEEN $starting_record AND $rec_pross order by tbl_id

 

Now the real kicker is that it reads the whole dataset with no problems however, it still runs after the dataset has been read (basically it is stuck..) Any ideas in what would be the proper SQL statement for this type of operation??

 

Thanks!
Kevin DavisKevin Davis

Never mind it is working.