Sign Up ›

Browse by Topic
ShowAll Questionssorted byDate Posted
+ Start a Discussion
ganeshjujjuruganeshjujjuru 

Can we Prevent duplicates from Dataloader

Is it possible to prevent duplicates using dataloader ,if it is possible then how can we prevent duplicates using Dataloader.... Please help me any body...?

Cory CowgillCory Cowgill

You cannot directly use Dataloader to prevent duplicates.

 

You can do things like install a DupeBlocker tool (Look on AppExchange) which can be configured to prevent duplicate records from being insterted into SFDC.

 

You could also use Excel to Highlight / Remove Duplicate Records before sending to CSV for Dataloader to consume.
liron169liron169
You can define unique field.

Then when you will try to insert record with the same unique field, it will automatically failed.
Ashish_SFDCAshish_SFDC

Hi Ganesh, 

 

Can how are you trying to use data loader for this, while Uploading?

 

Regards,

Ashish
Balakrishna N 25Balakrishna N 25
Dataloader does not inherently check for duplicates. There are a few options
1. Export data first if doing an update, and work with the exported list using VLOOKUPs in excel, etc.
2. Set a unique identifier on the object that can also be used in Dataloader. For contacts you might use a custom Email field for example. I’ve done this using concatenations of fields to create a unique identifier for bulk loads. Something like Field 4 = Field 1 & Field 2 & Field3, and set that as the unique identifier.
3. See if you can use data import wizard instead. I believe you can check for duplicates using that, but you do have less flexibility, and the 4 million records would be an issue.
Manuel Bustos 9Manuel Bustos 9
I use data loader and it detects duplicates, however as it loads in batches, if there are any duplicates in one batch it does not detect it. So, you have to put in data loader settings batch size=1 and then do the load, every record is going to be checked according to duplication rules.
Rajshree ManeRajshree Mane
Hi ganeshjujjuru,
You can try with trigger,  for example i have used account and in account i'm trying to prevent duplicate phone to insert.
​​​​​​​
trigger avoidduplicatephone on Account (before insert) {
    Set<String> SetPhone = new Set<String>();
    Map<String,Account> SetPhoneMap = new Map<String,Account>();
    Set<String> DuplicatePhone = new Set<String>();
    Map<String,Account> AccPhoneMap = new Map<String,Account>();
    List<Account> AccList = [Select id, name, phone from Account];
    if(AccList.size()>0){
        for(Account ac : AccList){
            AccPhoneMap.put(ac.Phone, ac);
        }
    }
    for(Account acc : trigger.new){
      if(!SetPhone.add(acc.Phone)){
          acc.Phone.addError('You cannot insert account with the same phone');
        }
        if(AccPhoneMap.containsKey(acc.Phone)){
            system.debug('AccPhoneMap'+AccPhoneMap);
             acc.Phone.addError('You cannot insert account with the same phone');
        }
   }
}