function readOnly(count){ }
Starting November 20, the site will be set to read-only. On December 4, 2023,
forum discussions will move to the Trailblazer Community.
+ Start a Discussion
sfdc-Lsfdc-L 

Pls need some help for the below scenario

Hi,

i am facing issue as given below scenario.
Current functionality is uploading the data in to custom object by using custom page i.e. 5k records,its working fine.
Requirement is need to upload 10k records,but when i was changed the query limit to 10000,getting below error

"Too many code statements: 200001
Error is in expression '{!readFile}' in component <apex:commandButton> in page svc_bidassetcsvuploadutility

An unexpected error has occurred. Your development organization has been notified."

i belive its throwing the error in insertBIdAssetRecords method but not able to find where do i optmaized the code,so here i am posting my code and i request you to verify it and make any changes if needed.Help is greatly appreciated

VF Page
---------
<apex:page sidebar="false" controller="Svc_BIdAssetCsvUploadUtility" tabStyle="B_SerialNumber__c">
<apex:form >
<apex:sectionHeader title="Upload data Assets"/>
<apex:pagemessages />
<apex:pageBlock >
<apex:pageblocksection columns="1" title="Upload CSV">
<apex:pageBlockSectionItem >
<font color="red"> <b>Please use the standard template to upload Id Asset records. <a href="{!templateUrl}" target="_blank"> Click here </a> to download the template. </b> </font>
<br/> <b>Maximum {!RECORD_PROCESSING_LIMIT} records can be inserted in every load.</b>
</apex:pageBlockSectionItem>
<apex:pageBlockSectionItem >
<apex:inputFile value="{!fileContentBlob}" filename="{!fileName}" accept="csv" alt="Browse the file to upload"/>
</apex:pageBlockSectionItem>
<apex:pageBlockSectionItem >
<apex:commandButton action="{!readFile}" value="Upload" id="theButton" style="width:70px;"/>
</apex:pageBlockSectionItem>
</apex:pageblocksection>
<apex:pageblocksection columns="1" title="Results" rendered="{!NOT(ISNULL(insertedRecords))}">
<apex:pageBlockSectionItem >
<apex:outputText style="font-weight:bold" value="Success: {0} records">
<apex:param value="{!successCount}"/>
</apex:outputText>
</apex:pageBlockSectionItem>
<apex:pageBlockSectionItem >
<apex:outputText style="font-weight:bold" value="Error: {0} records">
<apex:param value="{!errorCount}"/>
</apex:outputText>
</apex:pageBlockSectionItem>
<apex:pageBlockSectionItem >
<apex:outputPanel ><b>For more details download results</b><apex:commandLink action="{!downloadCsv}" value="here" id="theCommandLink"/></apex:outputPanel>
</apex:pageBlockSectionItem>
</apex:pageblocksection>
</apex:pageBlock>
</apex:form>
</apex:page>
---------------------------------------------------
Controller:
/* ==============================================================================
Name : Svc_BIdAssetCsvUploadUtility
Description : Controller class to reads a CSV file and insert Id-Asset records
================================================================================= */
public class Svc_BIdAssetCsvUploadUtility{
private List<ResultWrapper> displayResults;
private String tempalteName = 'BId-Asset Template';
private String templateId;
public transient String fileName{get;set;}
public transient Blob fileContentBlob{get;set;}
public transient Integer errorCount{get;set;}
public transient Integer successCount{get;set;}
public Integer RECORD_PROCESSING_LIMIT{get{return 10000;} private set;}

/*==========CONSTRUCTOR==========*/
public Svc_BIdAssetCsvUploadUtility(){}

/*=============================================================================================
Description : Method to read data from uploaded CSV file and then insert Id-Asset records.
===============================================================================================*/
public Pagereference readFile(){
try{
errorCount = 0;
successCount = 0;
displayResults = new List<ResultWrapper>();

if(fileContentBlob == Null || fileName == Null){
String errorStr = 'Please select a CSV file.';
ApexPages.Message errormsg = new ApexPages.Message(ApexPages.severity.ERROR,errorStr);
ApexPages.addMessage(errormsg);
return null;
}
if(!fileName.endsWithIgnoreCase('.csv')){
String errorStr = 'Only CSV files can be processed.';
ApexPages.Message errormsg = new ApexPages.Message(ApexPages.severity.ERROR,errorStr);
ApexPages.addMessage(errormsg);
return null;
}

String fileContentStr = fileContentBlob.toString();
List<List<String>> allRecords = parseCSV(fileContentStr,true);

if(allRecords.size() == 0){
String errorStr = 'Uploaded file contains no records.';
ApexPages.Message errormsg = new ApexPages.Message(ApexPages.severity.ERROR,errorStr);
ApexPages.addMessage(errormsg);
return null;
} else if(allRecords.size() > RECORD_PROCESSING_LIMIT){
String errorStr = 'You are trying to process ' + allRecords.size() + ' records. Maximum ' + RECORD_PROCESSING_LIMIT + ' records can be processed.';
ApexPages.Message errormsg = new ApexPages.Message(ApexPages.severity.ERROR,errorStr);
ApexPages.addMessage(errormsg);
return null;
}


Map<String, String> mapAssetPartNameToId = fetchAssetIds(allRecords);
insertBIdAssetRecords(allRecords, mapAssetPartNameToId);

} catch (Exception e) {
ApexPages.Message errormsg = new ApexPages.Message(ApexPages.severity.ERROR,e.getMessage());
ApexPages.addMessage(errormsg);
}
return null;
}

/*=============================================================================================
Description : Method to insert Id-Asset records.
@params : List<List<String>> allRecords - comma separated values for uploaded records
@params : Map<String, String> mapAssetPartNameToId - Map with Key=>AssetName and Value=> AssetId
@return : void
Modification History:
===============================================================================================*/
private void insertBIdAssetRecords(List<List<String>> allRecords, Map<String, String> mapAssetPartNameToId){
String partAssetName;
displayResults = new List<ResultWrapper>();
List<BId_SerialNumber__c> bId_AssetsToInsert = new List<BId_SerialNumber__c>();

for (Integer count = 0; count < allRecords.size(); count++){
List<String> fieldValues = allRecords.get(count);

ResultWrapper wrapper = new ResultWrapper();
wrapper.serialNumber = fieldValues.get(0).trim();
wrapper.pdtName = fieldValues.get(1).trim();
wrapper.bId = fieldValues.get(2).trim();

B_Id__c tempBId = new B_Id__c();
if(fieldValues.get(2).trim().isNumeric()){
tempBId.B_Id__c = Integer.valueOf(fieldValues.get(2).trim());
} else {
wrapper.notes = 'Id should be a numeric value';
}

BId_SerialNumber__c record = new BId_SerialNumber__c();
record.Asset__c = mapAssetPartNameToId.get(fieldValues.get(0).trim() + ',' + fieldValues.get(1).trim());
record.B_Id__r = tempBId;

bId_AssetsToInsert.add(record);
displayResults.add(wrapper);
}

if(bId_AssetsToInsert.size() > 0){
List<Database.SaveResult> saveResults = Database.insert(bId_AssetsToInsert, false);

Integer index = 0;
for(Database.SaveResult result : saveResults){
if(result.isSuccess()){
displayResults.get(index).bIdAst = result.getId();
displayResults.get(index).status = 'Success';
successCount++;
} else {
//Avoid over writing the custom error messages
if(String.isBlank(displayResults.get(index).notes)){
displayResults.get(index).notes = result.getErrors().get(0).getMessage();
}
displayResults.get(index).status = 'Error';
errorCount++;
}
index++;
}
}
}

/*=============================================================================================
Description : Method to query Asset records based on the Serial Numbers
===============================================================================================*/
private Map<String, String> fetchAssetIds(List<List<String>> allRecords){
Map<String, String> mapAssetPartNameToId = new Map<String, String>();
String baseQryStr = 'SELECT Id,Product2.Name,SerialNumber FROM Asset WHERE SerialNumber IN ';
String whereClause = '';

//Rows with BLANK values should not be included in query
for(List<String> fieldValues : allRecords){
if(String.isNotBlank(whereClause) && String.isNotBlank(fieldValues.get(0).trim())){
whereClause += ',\''+ fieldValues.get(0).trim() + '\'';
} else if(String.isNotBlank(fieldValues.get(0).trim())){
whereClause = '\''+ fieldValues.get(0).trim() + '\'';
}
}

String qryStr = baseQryStr + '(' + whereClause + ')';
List<Asset> listAssets = Database.query(qryStr);
for(Asset ast : listAssets){
mapAssetPartNameToId.put((ast.SerialNumber.trim() + ',' + ast.Product2.Name.trim()), ast.Id);
}

return mapAssetPartNameToId;
}

/*=============================================================================================
Description : Method to download load result in an excel file
===============================================================================================*/
public Pagereference downloadCsv(){
Pagereference pgRef = new Pagereference('/apex/Svc_BIdAssetCsvUploadResults');
pgRef.setRedirect(false);
return pgRef;
}

/*=============================================================================================
Description : Method to create url that points to standard template to load Id-Asset records
===============================================================================================*/
public String getTemplateUrl(){
If(templateId == Null){
List<Document> listDocuments = [Select Id from Document where Name = :tempalteName];

If(listDocuments.size() == 1){
templateId = listDocuments.get(0).Id;
}
}

return URL.getSalesforceBaseUrl().toExternalForm() + '/servlet/servlet.FileDownload?file=' + templateId;
}

/*=============================================================================================
Description : Method to prepare data to download in excel
===============================================================================================*/
public List<List<ResultWrapper>> getInsertedRecords(){
if (displayResults != null && displayResults.size() > 0){
List<List<ResultWrapper>> listResultLists = new List<List<ResultWrapper>>();
List<ResultWrapper> listResults = new List<ResultWrapper>();

ResultWrapper header = new ResultWrapper();
header.serialNumber='<b>Serial Number</b>';
header.pdtName='<b>Product Name</b>';
header.bId='<b>BId</b>';
header.bIdAst='<b>BIdAsset</b>';
header.notes='<b>Notes</b>';
header.status='<b>Status</b>';

listResults.add(header);
Integer count = 1;
for(ResultWrapper result : displayResults){
listResults.add(result);
count++;
if(Count == 1000){ //SFDC limit to display 1000 results on a visualforce page
listResultLists.add(listResults);
listResults = new List<ResultWrapper>();
count = 0;
}
}

if(listResults.size() != 0){
listResultLists.add(listResults);
}
return listResultLists;
}else {
return null;
}
}

=============================================================================================
Description : Method to parse CSV data
Comma separated values of uploaded CSV
===================================================
public static List<List<String>> parseCSV(String contents,Boolean skipHeaders) {
List<List<String>> allFields = new List<List<String>>();

// replace instances where a double quote begins a field containing a comma
// in this case you get a double quote followed by a doubled double quote
// do this for beginning and end of a field
contents = contents.replaceAll(',"""',',"DBLQT').replaceall('""",','DBLQT",');
// now replace all remaining double quotes - we do this so that we can reconstruct
// fields with commas inside assuming they begin and end with a double quote
contents = contents.replaceAll('""','DBLQT');
// we are not attempting to handle fields with a newline inside of them
// so, split on newline to get the spreadsheet rows
List<String> lines = new List<String>();

lines = contents.split('\n');

Integer num = 0;
for(String line: lines) {
// check for blank CSV lines (only commas)
if (line.replaceAll(',','').trim().length() == 0) continue;

List<String> fields = line.split(',');
List<String> cleanFields = new List<String>();
String compositeField;
Boolean makeCompositeField = false;
for(String field: fields) {
if (field.startsWith('"') && field.endsWith('"')) {
cleanFields.add(field.replaceAll('DBLQT','"'));
} else if (field.startsWith('"')) {
makeCompositeField = true;
compositeField = field;
} else if (field.endsWith('"')) {
compositeField += ',' + field;
cleanFields.add(compositeField.replaceAll('DBLQT','"'));
makeCompositeField = false;
} else if (makeCompositeField) {
compositeField += ',' + field;
} else {
cleanFields.add(field.replaceAll('DBLQT','"'));
}
}

allFields.add(cleanFields);
}
if (skipHeaders) allFields.remove(0);
return allFields;
}

//Wrapper class representing result of inserted records.

public class ResultWrapper{
public String serialNumber{get;set;}
public String pdtName{get;set;}
public String bId{get;set;}
public String bIdAst{get;set;}
public String notes{get;set;}
public String status{get;set;}
}
}
I would look forward to your reply... :)

Vinita_SFDCVinita_SFDC

Hello,

 

This error is usually caused by a combination of nested loops and large record sets. 

 

Possible Workarounds:

1)      @future method - Extract out some of the logic into @future methods so they can execute in a separate context.

2)      Move statements out of For Loops - Move statements out of the For Loops so they aren't executed as frequently

3)      Reduce/Combine statements - Do a single statement to get multiple values.  Use lists to do a single update instead of individual updates.

4)      Reduce the number of records processes - Reduce the batch size involved or artificially introduce a batch mechanism.

 

Troubleshooting:

1)      Insert System.debug(Limits.getLimitScriptStatements()); at the beginning of the code section to see the current limit.

2)      Insert System.debug(Limits.getScriptStatements()); at strategic areas of the code to identify the problem area.  This will usually be just outside of major loops and inside suspect loops.

3)      Look for areas where there's large statement usage and focus on clean that area up.

sfdc-Lsfdc-L

Thanks Vinit for the reply,

 

Could you pls help me out to make the changes in the code that would be really great.

 

Thanks in advance.