All about Upsert and External ID in Dataloader and Apex – Videos

You may be wondering that there are tons of articles available for upsert operation and what is need of one more blog post. I wanted to note everything important about upsert operations using datalaoder and Apex with identified gotchas at single place, and that’s why its here ūüôā .

External ID is field in Salesforce to say that this field is primary key in external Database which can be used to identify that external record exists in Salesforce or not ? There are many benefits of external key like :

  • It can be used to identify if record exists or not and record automatically inserted or updated using upsert operation
  • Upsert operation is supported by Dataloader, Apex and API
  • Any field marked as external id is indexed and can be used in SOQL or report filter
  • You can create Parent child record in¬†single statement in Apex

Using Upsert operation to create/update child record and relate to existing parent record with help of External Id

Lets assume that you have list of contacts and Account in external SQL Server database and you want to import those Account and¬†contact weekly. It is possible that you have already imported contacts previously¬†so don’t want to duplicate records¬†if already created in Salesforce. To solve this problem we can use external Id on Contact and use upsert operation on dataloader. Let’s assume external Id field on Contact is “SQL_Server__Id__c” (I am assuming that Account records in Salesforce also have external Id and records are already imported. Same below¬†steps can be used to import Account record as well).

  • Launch Dataloader installed from Salesforce and make sure settings¬†like “Batch Size”, “Server host”, “Start at row” are correct.
Salesforce Dataloader Settings
Salesforce Dataloader Settings
  • click on “Upsert” button.
  • Login Window may appear, so enter your Username and Password+Security token.
  • Select “Contact” object in next window and choose CSV file of contact you want to import in Salesforce.
  • It will alert informing total number of records going to be processed, click OK.
  • In next screen , you will be prompted to select field from Contact which should be used by Salesforce to identify¬†record and take decision that it needs to be updated or inserted. Id field and all fields which are marked as external Id will be listed here.¬†¬†Don’t expect this screen in any other operations except upsert.
External Id choice in Upsert operation of Dataloader
External Id choice in Upsert operation of Dataloader
  • As shown in above image, select correct external Id field (in this case SQL_Server_Id__c) and click next.
  • Next screen will show list of all Parent Object and each object will provide choice to select either Id field or external id field of Parent Object. This is one of amazing feature of Salesforce where Parent child relationship can be established with help of external Id (No need to use excel sheet Vlookup functions to extract 15 digit Salesforce Id to establish relationship). In this case example, we only want to create relationship with ¬†Account so we will use only Account’s external Id and leave others unselected.
Create Parent child relationship using External Id in Dataloader
Create Parent child relationship using External Id in Dataloader
  • At end of this article, I have provided Youtube Video link following same steps.
  • In Next window we need to map CSV file field with Salesforce fields. One point to notice in below image, how we are mapping Parent Account’s External Id with contact record.
Field Mapping to create Parent Child relationship using ExternalId in Salesforce
Field Mapping to create Parent Child relationship using ExternalId in Salesforce
  • Once all of above steps are followed, we can click on Next window and upsert operation will start. Salesforce will provide CSV file of success and failure results to perform¬†post analysis of operation.

How to use upsert operation in Apex

We can upsert records in Apex also, as shown in below code snippet

List lstContact = new List();
Contact con = new Contact (lastName = 'Zaa', SQL_Server_Id__c='3',firstName='Jitendra');
lstContact.add(con);
//.. Other Contact records added in List
upsert lstContact  SQL_Server_Id__c;

Above code snippet will perform upsert operation on all Contact with field ‘SQL_Server_Id__c’. If any one record will fail then none of contact record will be upserted. You may want to perform partial operation where if any record fails then it should not rollback other records processing. In this scenario Database.upsert comes to rescue. Below code snippet shows how we can use Database.upsert and how to iterate through errors.

List<Contact> lstContact = new List<Contact>();
Contact con = new Contact (lastName = 'Zaa', SQL_Server_Id__c='3',firstName='Jitendra');
lstContact.add(con);
//.. Other Contact records added in List
Database.UpsertResult[] results = Database.upsert( lstSGAccOppInsert, Contact.SQL_Server_Id__c.getDescribe().getSObjectField() ,false ) ;

for(Integer i=0;i<results.size();i++){
    if (!results.get(i).isSuccess()){
        Database.Error err = results.get(i).getErrors().get(0);
        System.debug('Error - '+err.getMessage() + '\nStatus Code : '+err.getStatusCode()+'\n Fields : '+err.getFields());
    }
}

How to insert parent and child record in single statement in Apex using External Id

As informed earlier in this post, we can use external Id of Parent record to insert parent and child in Same statement

//Create instance of Child record
Contact con = new Contact (lastName = 'Zaa', SQL_Server_Id__c='3',firstName='Jitendra');

//Create instance of Parenr record and only specify External Id, No Other fields
Account accountReference = new Account( Account_External_Id__c='21');                

//relate Child record with Parent
con.Account = accountReference;

Account parentAccount = new Account( Name='Cognizant', MyExtID__c='21');  

// Create the account and the Contact.
Database.SaveResult[] results = Database.insert(new SObject[] { parentAccount, con });

 

What if there are more than one records in Salesforce with Same external Id ?

In this case Dataloader or Apex will throw an error saying “Duplicate external id specified”.

Making external Id as a unique 

While trying to create External Id field, we have option to select whether field should be unique or not. If field already exists and we want to change it as unique external Id then it must be populated with unique values before making it unique.

Upsert using Command line Dataloader

You can read this post to know more about how to configure command line dataloader. For upsert, in config file “process-conf.xml” you need make below entry.

<entry key="sfdc.externalIdField" value="Master__r.External_id__c" />

and in field mapping file below entry

Id=Id
Master__r.External_id__c=Master__r\:External_id__c

Above relationship mapping is only applicable for upsert operations.

Important consideration for Upsert operation where external Id is not unique

To use upsert operation where external Id field is not unique, user performing operation must needs to have “ViewAllData” or “ViewAllRecords” permission else below error will be thrown :

System.SecurityException: ViewAllData or ViewAllRecords required to access external id fields which do not have a unique index

 

Related posts

  • Reddy Penna

    Hey Jitendra,

    Nice post.I am new(fresher) to Salesforce. I have a requirement where there is weekly update to Accounts and contacts. The update contains new records and updates to existing records. I am thinking of using upsert operation in Apex Data loader. Correct me If I am wrong.

  • Vandana

    Hello,
    Thank you for a great post.
    In the second video here where you are able to import Contacts with relationship to Accounts using External ID, how did or what caused data loader to show “Account” in Step2b (i.e. Choose your related objects) while in the first case it only show “Reportsto” which we ignored. (At 2.41 min in the second youtube video). Is it because the csv file had Parent Account ID column? If yes should I be careful about what I name this column in the file?
    VK..

    • Hi Vandana,
      its because of 2 reason :
      1. I selected Upsert Operation
      2. There was external Id field in Parent object

      Thanks,
      Jitendra Zaa