As your Salesforce Organization undergoes heavy customization and frequent builds, moving changes from one Sandbox to other sandboxes starts taking longer time and effort. Also, in normal Salesforce project, there are chances that you will have minimum three sandboxes likely Developer Sandbox, QA Sandbox and UAT Sandbox. After some time you will be in need of some solution which can reduce your effort.
Almost 5 years back, when I was working on .net along with Salesforce, I posted article on Continuous Integration of ASP.net and subversion as a code management with MSBuild on code project.
This time its Salesforce using Jenkins. In this article I will walk through solution of above problem using Jenkins. Don’t forget to watch Video at end of this article, where I provided demo of everything explained in this article.
Prerequisite Software:
- ANT
- Salesforce Migration Tool
- Git
- Eclipse or PuTTYgen to generate SSH key
I am assuming you already know about below concepts:
- How to use Git with Salesforce
- Using ANT migration tool in Salesforce
- Generating SSH key
- Automated Daily Backup of Salesforce Using ANT Migration Tool and GIT (Optional)
I will suggest to get familiar with first three topics before jumping to Continuous Integration.
There are two ways to start working with Jenkins
- Install Jenkins on your local system
- Use Cloudbees online
Below diagram shows complete flow of Continuous Integration we are going to setup in this article.
For this tutorial, I will install Jenkins on my local system. To install it, Navigate to Jenkins homepage and download installer.
During installation, you may be prompted to install Jenkins as “Windows Service”, click yes for that option. Once installed navigate to “http://localhost:8080”, and you should be able to Jenkins homepage.
Installing Git plugin
By default Jenkins does not provide support for Git, so we will need to install it separately.
To install Git Plugin, Navigate to “Jenkins Homepage | Manage Jenkins | Manage Plugins | Available” and search for Git. From available list, install “Git Plugin”. I would suggest to install “Bitbucket Plugin” as well.
Installing chatter Plugin
Once Git plugin is installed, we will need to install chatter plugin as well from superfell’s github repository. Assuming you already have Maven installed on your system, download above repository as a zip and extract it on your system. After extract, run command “mvn” in that directory. It will create folder by name “target” and plugin file by name “ChatterPlugin.hpi” would be available.
If you don’t want to perform maven build then you can bypass above step and download file from here, however it is always suggested to build file from latest repository code. If you are downloading “ChatterPlugin.hpi” from my website then don’t forget to rename file to “ChatterPlugin.hpi”.
Once you have “ChatterPlugin.hpi” file on your system, you need to install it into Jenkins. Navigate to “Jenkins Homepage | Manage Jenkins | Manage Plugins | Advanced | Upload Plugin”. Select “ChatterPlugin.hpi” and upload it.
Salesforce migration ANT script
Now, we need to create “build.xml” to retrieve changes from one Org and Deploy to other Org. Assuming, you already know how to use ANT Migration tool I will directly show code snippet of my “build.xml” file.
<project name="Code Backup Without Folders" default="retrieve" basedir="." xmlns:sf="antlib:com.salesforce">
<property file="build.properties"/>
<property environment="env"/>
<target name="Deploy">
<sf:deploy
username="${Sandbox2.username}"
password="${Sandbox2.password}"
serverurl="${sfSandbox.serverurl}"
deployRoot="${Sandbox2.retrieveTarget}"
runAllTests="true" />
</target>
<target name="retrieve">
<sf:retrieve username="${Sandbox1.username}"
password="${Sandbox1.password}"
serverurl="${sfSandbox.serverurl}"
retrieveTarget="${Sandbox1.retrieveTarget}"
unpackaged="${Sandbox1.unpackaged}"/>
<echo message="Commiting all changes with message ${gt.commitMessage}" />
<git command="add" dir="${Sandbox1.gitDirectory}">
<args>
<arg value="." />
</args>
</git>
<git command="commit" dir="${Sandbox1.gitDirectory}">
<args>
<arg value="-am ${gt.commitMessage}" />
</args>
</git>
</target>
<macrodef name="git">
<attribute name="command" />
<attribute name="dir" />
<element name="args" optional="true" />
<sequential>
<echo message="Executing Command in folder @{dir}" />
<echo message="git @{command}" />
<exec executable="git" dir="@{dir}">
<arg value="@{command}" />
<args/>
</exec>
</sequential>
</macrodef>
</project>
Username, Passwords and other values are coming from “build.properties” file. As you can see in above code, Changes are fetched from “Sandbox1” and deployed to “Sandbox2”.
Setting-up Bitbucket account
If you are following my blog, you may already know that I give preference to Bitbucket over Github because it allows me to setup private repository free of cost.
First you will need to generate SSH key for authentication, for this you can either use Eclipse or PuTTYGen. In Video demo of this article, i have used PuTTYGen. Once SSH key is generated, upload Public key in Bitbucket by navigating to “Manage Account | Securtiy | SSH Keys | Add Key”. Save Private key with extension “ppk” somewhere on your local system.
You will need to commit all your changes of “Sandbox1” to this Bitbucket account so that it can be deployed to “Sandbox2”. For this you can either use Eclipse or TortoiseGit.
Create new project in Jenkins
Navigate to “Home page | New Item”. Give appropriate item name and then choose “Freestyle project”.
In “Source Code Management”, choose “Git”. In Repository URL, provide repository URL of Bitbucket for SSH protocol. In Credentials, click on Add, choose “SSH Username with private key”. Provide username and paste private key saved in “ppk” file. If there is any Passphrase then provide it and save it.
Build Triggers
There may be any combination which will cause Jenkins to trigger build. For example if code is pushed to your Git repository, or poll Git repository after some predefined interval to see if there is any change or run build periodically. You can have one or more than one combination to trigger builds. In this article I have selected “Poll SCM” option. It will check my git repository after every 15 minutes for changes and if it finds that repository has been changed then it will start building. If you select “Pol SCM”, then text box will appear where we need to provide crone statement to decide its scheduled time. I have provided “H/15 * * * *” which means, it will run every 15 minutes.
Build
In this section choose “Invoke Ant”. In Targets, provide Ant target names separated by space. provide location of folder where “build.xml” is present in text box “Build File”.
Post-build Actions
In this section of project configuration, we need to setup task that can be executed after build. There are number of options available, however we will choose “Chatter results”. This task will be available only if you have installed “Chatter plugin” as explained above. Here, we need to provide Username and Password to connect to Salesforce. If you want to connect to Sandbox or My Domain then need to setup value in text box “Override Login Server” else keep it blank.
After all above settings, save this information and we will be good to go. You can see log of attempt by Jenkins to determine if there is any change in git repository, it would be available at left menu by name “Git Polling Log”.
We can also manually trigger build by clicking on “Build Now” link visible in above image. All build results will be available below “Git Polling Log”.
Video Tutorial
Please share your feedback in comments section.
Hi Jitenda, I am getting the below error while connecting to my bitbucket by ssh.
Failed to connect to repository : Error performing command: /usr/local/git ls-remote -h git@bitbucket.org:suddeb/sudipta-personal.git HEAD
Can you please help me to understand the problem
Hi Sudipta, Did you tried to connect repository without jenkins first using SSH key ? I would suggest to try with Tortoise Git or Eclipse first. Let me know if you still face any issue.
Hi Jitendra, Finally it is working. My problem was with the ssh. Thanks for such a great post.
You can check if your Git is working without jenkins by this approach – https://www.jitendrazaa.com/blog/others/tips/getting-started-with-git-and-bitbucket-in-5-minutes-with-tortoisegit/
Failed to connect to repository : Error performing command: git.exe ls-remote -h git@github.com:makk550/test.git HEAD
please help
Hi Jitendra,
Thanks for the great post on such an important topic.
However I am facing an issue with these steps.
I am trying to run it via the command prompt itself first without using jenkins.
I am able to run the build.xml properly however I am getting a message after running the ‘retrieve’ target which says ‘nothing to commit’. However the build runs successfully without any issues.(I am able to retrieve the contents successfully from the source org)
I am not sure how to configure/pass (to git) the path of the folder where I am retrieving the contents from my source org. I have created a folder which I have configured as a git repository which I am passing as a parameter to Sandbox1.gitDirectory in your build.xml.
Thanks,
Nitin.
Hi Nitin,
In CI, first you have to retrieve metadata from Salesforce Org and then commit to GIT.
in your case, when you are retrieving from your org, No metadata information is changed and that why you are getting message saying “Nothing to commit”.
Hi Jitendra,
Many thanks for the post.
However, i am getting the below error when running my build. It seems Jenkins is not able to clone the remote Git repository,
ERROR: Error cloning remote repo ‘origin’
Waiting for your response.
Thanks,
Ankit Kumar
Hi it is very helpful, I just have a question, there is a way just for take what has changed and not the entire package.xml that include everything, I mean if I just add a field and salesforce for example, and then I push a commit with this change, its possible just take that change and deploy it? Thanks,
Hey Edgar, Thats something very advance and cannot done by Jenkins. However wait for my next post on solution of this, 1 or 2 day 🙂
Hi Jitendra,
Thank you for the wonderful post.
Is there any other way we can achieve what Edgar was asking for?
Yes, I have developed a solution for that and will post here very soon.
Thank you.
I will wait for your post.
can you help me with the above solution?
Hi Jitendra,
Could you please share link for solution for this question, if its available here
Hi Jitendra,
Thank you for the blog. It helped me to automate my migrations. Can you share the solution with us that you have found..
Thank you
Hi Edgar, did you find a solution for this issue. I think the problem is that you dont want to deploy all the contents of the repo, just the changes which would require using git diff, moving all the files, building a package.xml and deploying it to an org using ant.
that make sense, I didn’t finish to implement this, there are many manual actions, so yes, diff would be the option to generate the package xml and I was thinking in metadata api to complement the process. Thanks,
Great post…. JIC now that JenkinsChatter plugin is able to publish the code coverage percent also.
E.G.
Build: SForce-CI-Project-job 2 is SUCCESS
Coverage Result: 76.91% of code coverage, Risk status.
Coverage Status: Danger (0% – 74%): 298 files. Risk (75% – 79%): 84 files. Acceptable (80% – 94%): 267 files. Safe (95% – 100%): 277 files.
Test Result: 0 tests failing out of a total of 3,276 tests.
Thanks for information, will look into this,
In this demo, your source is Salesforce Sandbox, your destination is another sandbox. How are you tagging commits to specific branch in Github. If you can explain, end-to-end flow in video, could boot it more. Induce a change in org, show the commit in github, then show the change in second org after successful deploy
Also, I am inquisitive to know, how about committing data in github after approval of pull request can trigger a deployment to destinations org(s).
How do you generate package.xml for continuous integration?
I do it in multiple ways depending on my projects. We can manually keep updated Package.xml on build branch or using this blog post https://www.jitendrazaa.com/blog/salesforce/auto-generate-package-xml-using-ant-complete-source-code-and-video/ . If you can share your problem, we can come up with solution.
I am getting login to fail error while trying to retrieve the package from developer org using jenkins
The same set of file works perfectly using command line.Can you help me here?
You are not able to make internet connection. Are you using some proxy, if yes then proxy settings needs to be done in ANT.
I am not using any proxy.But if tried to retrieve the files using the command line its working fine.
Hi Jitendrazza, thanks for sharing knowledge. I tried to PoC the retrieve and deploy concept but I failed. I got those erros messages below:
1. The flows will only deploy in deactivated mode in the target org by default. They have to be manually activated in the target org after deployment.
2. objectTranslations/Case-pt_BR.objectTranslation — Error: insufficient access rights on cross-reference id
What is the best way to solve this issue? I’m trying to set a CI machine with the less human interation possible. Could you help me? Regards, Felipe from Brazil
Once you retrieve metadats from Salesforce, you will need to perform some extra steps. You can create custom “target” in ANT to perform text replacement in files. Example – to deactivate flow while deployment, you can replace contenet in XML using replace task – https://ant.apache.org/manual/Tasks/replace.html
Thanks for the response, Jitendrazza! I will apply some replacement rules and test it again. I’m implementing Continuous integration on the company who I’m in governance area. We have a complex parallel development context and it should solve almost all issues we have. If you have some known issues please share with me. Regards.
Hey Jitendra. I’m back again. I need help to design the process about using repository with salesforce. I’ve built all the ant jobs, I tried to do with deployRoot and zipfile(using ant task to zip the workspace) deploy but didn’t work. My PoC consists in develop in a Sandbox A commit to a branch( with jenkins retrieve and commit to bitbucket job), merge with master and then deploy to Sandbox SIT ( with jenkins checkout and deploy job). When I try to deploy the check out from the master with a delimited package.xml I got many errors about the files are not specified in package.xml. But I need to deploy only what is wrote in this manifest, not all workspace files. Can you brief me a process to fix that? Best Regards, Felipe from Brazil
@JitendraZaa:disqus
Hi Jitendra,
If there are multiple Developer sandboxes , then how the merging will happen on repository? For example same apex trigger is modified in 2 different orgs. Will the merging will automated or we have to do it manual?
Thanks,
Nitish
Hi Jitendra,
This is really very helpful . I am getting error while setting bitbucket repository SSH url under jenkins project . Could you please help me to resolve this .
Error – Failed to connect to repository : Error performing command: git.exe ls-remote -h git@bitbucket.org:rampgenkins/testdeployment.git HEAD
Thanks,
Ram
Hi Ram,
Any update on this issue? I am facing the same issue. Thanks in advance.
Hi Jitendra,
We are trying to implement Continuous Integration between our Dev & Sandbox orgs. We successfully synced Local (Eclipse) & Remote repo(Bitbucket). We set up a Cron in Jenkins that polls remote repo every 15 mins. So far everything seems to be working fine.
When we try to build the project in Jenkins, the build fails as below:
https://uploads.disquscdn.com/images/e9004b4b995ffdab39a6339d8d6dafe49f31712189e016bb9f59a2709fe2ffe6.png
We have used the similar build.xml & build.properties as shown in this blog. Kindly assist.
Thanks!
Good Day Jitendra,
Few questions where i need your expert advice:
1) Best approach for code reviews in Salesforce CI/CD pipelines? We are trying to plug Crucible/FishEye in to our Tools with Bitbucket as the central repository.
2) How do we establish a pipeline job in Salesforce for Unit tests? Can we run Junit in salesforce or all the unit tests are written internal to salesforce code?
Thanks!
Hi Jitendra,
Thank you for the blog. It helped me to automate my migrations. I have another requirement which needs your guidance. After i push to github I am trying to create the ant script so that it will push only changed components from org 1 to org 2. Please give your thoughts on this.
Thank you
Hi Jitendra,
when i enter my ssh/https url key in jenkins, it showing below issue
https://uploads.disquscdn.com/images/c5f3e295f9bae2ba2a57b5e6d567a3b7054239161d62f0558432acb07ec33d38.png
Hi Jitendra,
I want metadata to be moved from Bitbucket repo to SFDC org. If my understanding is not wrong, The Build.xml being referred pulls metadata from SFDC Org1 to SFDC Org2. In my case, How can i let Ant script know my bitbucket path/credentials, so Ant script can pull my data and push in SFDC org.
Any help will be appreciated.
Thanks,
Abhishek
Hi Abhishek… Any update for your issue as i am facing same..
Any help will be appreciated Abhishek and Jitendra…
Thanks,
Ashish Jain
Hi Jitendra
I am implementing continues integration with git and jenkins to salesforce i am getting some errors can you help me on this
BUILD FAILED
build.xml:67: The following error occurred while executing this line:
build.xml:86: The directory D:TEKMobile MiniCode RepositoryCPCDEVCPCDEV_MMI${Sandbox1.gitDirectory} does not exist
Total time: 3 minutes 53 seconds
Build step ‘Invoke Ant’ marked build as failure
Finished: FAILURE
Seems your script not able to resolve property – ${Sandbox1.gitDirectory}
Thanks for the reply
Which value we need to pass in this variable {Sandbox1.gitDirectory }.
Can you please help me on this
Which value we need to pass in this variable {Sandbox1.gitDirectory }.
Hi Jitendra,
Hi it is very helpful, I just have a query, there is a way just for take what has changed and not the entire package.xml that include everything, I mean if I just add a field and salesforce for example, and then I push a commit with this change, its possible just take that change and deploy it?
Thanks
Ashish
Yes. I have built a plugin but cannot share outside. However, You can have a look on this Git repository. I have not tried it yet, but seems promising – https://github.com/aesanch2/salesforce-migration-assistant
Thanks Jitendra Zaa !! Is this paid plugin. If yes, then please share the cost of this plugin.
I have reviewed above link but not getting point. Could you please give me more details?.
1. I am using Ant for build.
2. I have integrated jenkins to bitbucket.
3. when user click on merge button in bitbucket then jenkins started build.
Query: As jenkins getting code from local so it will deploy all the code(commit and without commit). Is there any way so we can get code from bitbucket not from local. Is there any specific configuration in build.xml or package.xml or jenkins?
I want only commit code.Please share your thought.
Thanks
Ashish
any updates ?
Hi Jitendra
I had a question on the continuous integration workflow – why is it that the workflow is set to pull the changes from the salesforce org as opposed to just pulling from the repository or is it just a different usecase?
I used your workflow to implement the same integration workflow only using svn instead of git; but modified the ant build to retrieve the latest changes from the svn repository as opposed to the salesforce org – the expectation being that when a developer checks in code to the svn; the other developers subscribed to the same repository get the code updated in their local as well as developer orgs from the repository as opposed to the source org- which may or may not have the changes pushed to it yet from the repository for whatever reason.
So the flow looks something like this https://uploads.disquscdn.com/images/e2709d560b68089f31b64516add3d395c909b03542983b3f2d0d9a8eef5d0297.png
Let me know if this approach is incorrect or has any issuesbefore I share it with my team.
Thanks a lot.
hai jitendra
thanks for a great blog. i tried the exact steps as said here, but the build seems to always a failure.
this is the error that i am getting.
ERROR: Could not connect to SMTP host: localhost, port: 25
javax.mail.MessagingException: Could not connect to SMTP host: localhost, port: 25;
nested exception is:
java.net.ConnectException: Connection refused: connect
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1934)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:638)
at javax.mail.Service.connect(Service.java:295)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at javax.mail.Transport.send0(Transport.java:194)
at javax.mail.Transport.send(Transport.java:124)
at hudson.tasks.MailSender.run(MailSender.java:131)
at hudson.tasks.Mailer.perform(Mailer.java:171)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1752)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:286)
at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:231)
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1900)
… 18 more
Finished: FAILURE
can u please help me here?
Hi Jitendra…mayi know what should i pass to “${Sandbox1.gitDirectory}”? ..thanks
It would be path of Git folder , if you want to automated Git commit and push.
thanks for reply Jitedra. Iam trying to deploy changes from DEV to UATPROD.. Iam polling my DEV and retrieving components from DEV GIT FOLDER and trying to deploy same components to UATPROD GIT FOLDER with Commit Push.. When i kept my gt.gitDirectory = C:\GIT REPO\UATPROD in properties file , i see error C:\Salesforce ANT\GIT REPOUATPROD does not exist.. How can ensure i give right path to my UATPROD Git Folder so that push+commit happens there?.. my ANT location is C:\Salesforce ANT.. thanks for your time
If you are sure that your path is correct, then try to replace backslash by double backslash. So, path would become “C:\\Salesforce ANT\\GIT REPOUATPROD”
Hi Jitendra,
I have used the Git folder path in place of “${Sandbox1.gitDirectory}” still it was throwing the error saying it does not exist.
Can you help me out or Can you please provide the Build.properties file if possible?.
thanks Jitendra, appreciate all your contribution . I will try .
How to push data to git using Ant build.xml
sir how to send data to git hub when i retrieve data from salesforce
Hello,
I am pushing the code in the git and then from git i have taken the code in the Jenkins now i want to push the code into the salesforce dev org 2. Using the salesforce-migration assistant plugin in the Jenkins can you help me out
Hi Jitendra, Great post.. However, I want to ask , what if the build.property file Salefsorce credentials have to be stored in a secure way? How can we achieve this ?