+ Start a Discussion
Andrew TelfordAndrew Telford 

CPU Usage issue Sending Email via SingleEmailMessage()

I am having issues with the CPU limit when sending out emails. The ideal scenario is to be able to send out 1000 emails in a hit but if I try that it exceeds the limits of the CPU. I can send 150 a time which is good but not entirely the best option for use at this point.

I am sending it this way rather than via ExactTarget or similar as we don't currently have connections to these services and we are sending attachments to the recipients.

Any thoughts as to where I should start to reduce the CPU impact so that I can increase the number of emails that I can send in one schedule execution?

Screen shot of Log File

 
//------------------------------------------------------------------------------------------------------
    //--
    //--    send_mailout
    //--
    //--    Use:
    //--    This is the process to actually send the email and will be triggered by a scheduled APEX
    //--
    //--
    //------------------------------------------------------------------------------------------------------
    PUBLIC VOID send_mailout(STRING mailOutName, INTEGER iTotalSend)
    {
        //--    Our variables
        BOOLEAN sndSuccess = FALSE;
        
        strTemplate_Initial = mailOutName;
        strTemplate_Password = mailOutName + ' - Password';

        //--    Set the FROM email address
        //--	------------------------------------
        for(OrgWideEmailAddress owa : [SELECT id, Address, DisplayName FROM OrgWideEmailAddress WHERE DisplayName = 'CommInsure Communications']) 
        {
            strEmailFrom = owa.id;
        } 

        //--    Get the details of who we are going to send to
        //--	----------------------------------------------------
        objMailout = [SELECT Id, contact__c, contact__r.Name, recipient_first_name__c, recipient_last_name__c, caps__r.AccountID__c, file_password__c, email_address__c 
                      FROM ci_mailout__c 
                      WHERE Sent__c = FALSE 
                      AND Mailout_name__c = :strTemplate_Initial 
                      ORDER BY Policy_count__C Desc 
                      LIMIT :iTotalSend];
        FOR( ci_mailout__c mo : objMailout )
        {	mailoutIDs.add( mo.Id );	}

        //--	Create some attachment maps
        MAP<ID, String> mName = NEW MAP<ID, String>();
        MAP<ID, BLOB> mBody = NEW MAP<ID, BLOB>();
        MAP<ID, String> mContentType = NEW MAP<ID, String>();
        objAttachment = [SELECT ID, ParentID, Name, ContentType, body FROM attachment WHERE ParentId IN :mailoutIDs];
        FOR( attachment thisAttachment : objAttachment )
        {
            mName.put( thisAttachment.ParentID, thisAttachment.Name );
            mBody.put( thisAttachment.ParentID, thisAttachment.body );
            mContentType.put( thisAttachment.ParentID, thisAttachment.ContentType );
        }
        
        SYSTEM.debug(thisScript + 'Size of Mailout: ' + objMailout.size());

        //--    Get out templates
        //--	--------------------------------------------------------------------
        objTemplate_email = [SELECT id, Name FROM EmailTemplate WHERE Name = :strTemplate_Initial];
        FOR(EmailTemplate thistemplate_email: objTemplate_email)
        {
            template_email_id = thisTemplate_email.id;
            SYSTEM.debug(thisScript + 'Initial Template: ' + thisTemplate_email.Name);
        }

        objTemplate_password = [SELECT id, Name FROM EmailTemplate WHERE Name = :strTemplate_Password];
        FOR(EmailTemplate thistemplate_password: objTemplate_password)
        {
            template_password_id = thistemplate_password.id;
            SYSTEM.debug(thisScript + 'Password Template: ' + thistemplate_password.Name);
        }

        
        //--    Loop through the objMailout list and create the emails to be sent
        //--	--------------------------------------------------------------------
        FOR (ci_mailout__c thisMailOut: objMailout) 
        {
        	SYSTEM.debug(thisScript + 'Looping through the mailout data');
            Messaging.SingleEmailMessage email = new Messaging.SingleEmailMessage();
            email.setTargetObjectId(thisMailOut.contact__c);
            email.setTemplateId(template_email_id);
            email.setSaveAsActivity( TRUE );
            email.setToAddresses(new String[] { thisMailout.email_address__c });
            email.setTreatTargetObjectAsRecipient(FALSE);
            email.setUseSignature(FALSE);
            email.setWhatId(thisMailout.Id);
            email.setOrgWideEmailAddressId(strEmailFrom);

			//--    Get the Email Attachment
			//--	------------------------
            SYSTEM.debug(thisScript + 'No Attachments: ' + objAttachment.size());
            IF( mName.keySet().contains(thisMailout.Id) )
            {
                SYSTEM.debug(thisScript + 'Looping through Attachment');
                Messaging.EmailFileAttachment efa = new Messaging.EmailFileAttachment();
                efa.setFileName( mName.get(thisMailout.Id));
                efa.setBody( mBody.get( thisMailout.Id ));
                efa.setContentType( mContentType.get( thisMailout.Id ));
                email.setFileAttachments(new Messaging.EmailFileAttachment[] {efa});
            }

            //--	Add the email to the msgList for sending & keep track of the totalSend
            //--	----------------------------------------------------------------------------
            msgList.add(email);
            iCountTotalSend++;

            //--	Check and add the attachment if we need too
            //--	------------------------------------------------------------
            SYSTEM.debug(thisScript + 'Check if we have a password template');
            IF ( objTemplate_password.size() > 0 )
            {
                SYSTEM.debug(thisScript + 'Creating Password Email');
                Messaging.SingleEmailMessage email_pass = new Messaging.SingleEmailMessage();
                email_pass.setToAddresses(new String[] { thisMailout.email_address__c });
                email_pass.setTreatTargetObjectAsRecipient(FALSE);
                email_pass.setTargetObjectId(thisMailOut.contact__c);
                email_pass.setTemplateId(template_password_id);
                email_pass.setSaveAsActivity( TRUE );
                email_pass.setUseSignature( FALSE );
                email_pass.setOrgWideEmailAddressId(strEmailFrom);
                email_pass.setWhatId(thisMailout.Id);

                //--	Add the email to the msgList for sending & keep track of the totalSend
                //--	----------------------------------------------------------------------------
                msgList.add(email_pass);
                iCountTotalSend++;
            }

            //--	Now that we have created the email to be sent, let us update the details of the mailout so we can track once it is actuall sent
            //--	--------------------------------------------------------------------------------------------------------------------------------
            thisMailout.Sent__C = TRUE;
            thisMailout.Sent_Date__c = SYSTEM.now();
            updateMailout.add( thisMailout );

            //--	Check to see if we need to send a batch out because we are at the end
            //--	------------------------------------------------------------------------
            SYSTEM.DEBUG(thisScript + 'Message List size: ' + msgList.size());
            IF( msgList.size() == iBatchSize || objMailout.size() == iCountTotalSend )
            {
                List<Messaging.SendEmailResult> results = Messaging.sendEmail( msgList, false );
                IF(!results.get(0).isSuccess())
                {
                    SYSTEM.DEBUG(thisScript + 'This was unsuccessful: ' + results[0].getErrors().get(0).getMessage());
                    BREAK;
                }
                ELSE
                {
                    SYSTEM.DEBUG( thisScript + 'We have successfully sent a batch of emails. ' + iCountTotalSend + ' of ' + objMailout.size() );
                    msgList.clear();
    
                    //--	Now that we have update the send list so we don't send out again
                    //--	------------------------------------------------------------------------------------------------
                    TRY{ update updateMailout; }
                    CATCH ( Exception ex )
                    {	 SYSTEM.DEBUG(thisScript + 'Could not update the Mailout. Cause: ' + ex.getCause() );	updateMailout.clear();	}
                }
            }	//--	End Sending out the batches	
        }	//--	End Looping throught objMailout
    }	//--	End send_mailout

 
Alain CabonAlain Cabon
Hello Andrew,

You are clearly in a synchronous limit.  The asynchronous limit is much bigger (60,000).
https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_gov_limits.htm

How do you launch this class?

Using the scope of the querylocator over your "ci_mailout__c" object", you can treat many more emails by batches.

Your case is interesting. I wrote a batch for sending thousands of customized chatter messages (without attachments, ligther than your case) and I never had problems of CPU. 

Regards
Alain
Andrew TelfordAndrew Telford
Hi Alain

I launch the process from a scheduled Apex (ci_schedule_class).

I am in the process of amending to run the function as a @future ... see how that flies 
Andrew TelfordAndrew Telford
Well ... I managed to get around this issue by using @future for the particular class. This has allowed me to no send out 1000 emails in one batch and of course I can now schedule that and send out 5000 in a day via the API.

@Alain, are you able to clarify the following?
Using the scope of the querylocator over your "ci_mailout__c" object", you can treat many more emails by batches.

 
Alain CabonAlain Cabon
Andrew, so it is strange that your CPU limit is not equals to 60,000 milliseconds using a future method (?)

You are using queue which one of the technique for batches.

1) Can you know if this query is time-consuming. Perhaps not. 
objMailout = [SELECT Id, contact__c, contact__r.Name, recipient_first_name__c, recipient_last_name__c, caps__r.AccountID__c, file_password__c, email_address__c 
                      FROM ci_mailout__c 
                      WHERE Sent__c = FALSE 
                      AND Mailout_name__c = :strTemplate_Initial 
                      ORDER BY Policy_count__C Desc 
                      LIMIT :iTotalSend];

2) Many batches are based on a main query used as the query locator that permits to by-pass some governor limits but not the CPU limit for you.

Batch Apex Examples: The following example uses a Database.QueryLocator: Invoking Apex page 250, Apex Developer Guide Version 38.0, Winter ’17​
global class UpdateAccountFields implements Database.Batchable<sObject>{
    global final String Query;
    global final String Entity;
    global final String Field;
    global final String Value;
    global UpdateAccountFields(String q, String e, String f, String v){
        Query=q; Entity=e; Field=f;Value=v;
    }
    global Database.QueryLocator start(Database.BatchableContext BC){
        return Database.getQueryLocator(query);
    }
    global void execute(Database.BatchableContext BC,
                        List<sObject> scope){
                            for(Sobject s : scope){s.put(Field,Value);
                                                  } update scope;
                        }
    global void finish(Database.BatchableContext BC){
    }
}
page 252:
1) A maximum of 50 million records can be returned in the Database.QueryLocator object. If more than 50 million records
are returned, the batch job is immediately terminated and marked as Failed.
2) If the start method of the batch class returns a QueryLocator, the optional scope parameter of Database.executeBatch
can have a maximum value of 2,000.
If set to a higher value, Salesforce chunks the records returned by the QueryLocator into smaller
batches of up to 2,000 records. If the start method of the batch class returns an iterable, the scope parameter value has no upper
limit. However, if you use a high number, you can run into other limits

Your case is interesting because the profiling under the developer console didn't help you.
Alain CabonAlain Cabon
@andrew: 

1) Apex Developer Guide Version 38.0, Winter ’17​ page 231

Future Methods with Higher Limits (Pilot): Note: We provide this feature to selected customers through a pilot program that requires agreement to specific terms and conditions. To be nominated to participate in the program, contact Salesforce.
Modifier Description
@future(limits='2xCPU') CPU timeout is doubled (120,000 milliseconds).
@future(limits='3xCPU') CPU timeout is tripled (180,000 milliseconds).


2) The cache is also customizable.  But what can you put in your cache for your problem?  https://trailhead.salesforce.com/fr/module/platform_cache

Alain
Andrew TelfordAndrew Telford
Thanks for the responses Alain

I have managed to use the @future method successfully sending out two lots of 1,000 emails within 10 minutes of each other by exevuting manually.

In answer to your questions ...
1) Can you know if this query is time-consuming. Perhaps not. 
objMailout = [SELECT Id, contact__c, contact__r.Name, recipient_first_name__c, recipient_last_name__c, caps__r.AccountID__c, file_password__c, email_address__c 
                      FROM ci_mailout__c 
                      WHERE Sent__c = FALSE 
                      AND Mailout_name__c = :strTemplate_Initial 
                      ORDER BY Policy_count__C Desc 
                      LIMIT :iTotalSend];
I don't believe this query is taking too long and would have minimal impact to CPU - I believe. The fact that we are adding attachments I think is the bigger user of CPU.

I don't plan on running 50 million record processes anytime soon but I will look into the Batch process further. I did do some more looking after your initial response and will continue to do so.

thanks for your responses.


 
Alain CabonAlain Cabon
I never send thousands of emails with Salesforce with attachments so your Apex code is interesting. 
The governor limits are very constraining for the emails (Using the API or Apex, you can send single emails to a maximum of 5,000 external email addresses per day based on Greenwich Mean Time (GMT). So you reach yet the limits and your code works.
Using queue and future method are good techniques.

For the problem of attachments (time consuming), you think that the most time consuming part of the code is  objAttachment = [SELECT ID, ParentID, Name, ContentType, body FROM attachment WHERE ParentId IN :mailoutIDs];
The platform cache is far too small for the attachments (10 MB).

The other possibility would be to mix several SOQL queries in just one using the relationships.

Wait and see for new avenues of investigation which can help you here.