Continuous Integration With Salesforce

Salesforce is a market-leading Customer Resource Management (CRM) solution, running as a cloud-based multi-tenant application.  In fact, Salesforce continues to be the leader in Gartner's Magic Quadrant for Enterprise Application as a Service (Worldwide).

While Salesforce may lead the market in CRM functionality, the platform provides challenges to both application developers and DevOps staff.  This article will focus on the main struggle that DevOps teams often face — Continuous Integration with Salesforce.

The Issue

Salesforce is not a typical application from an architecture perspective.  The biggest difference from a J2EE or .NET application is how the program code is stored.  With Salesforce, a majority of the program logic is stored in XML files. These XML files are then used with the Salesforce Metadata API to deploy program code into an instance of Salesforce, often referred to as an "Org."

The problem with many source control systems is that they aren't the best at figuring out how to merge differences with XML files, especially when the ordering of the information within the files changes between source Orgs.

The Challenges

There are a few challenges that DevOps teams must consider with the Salesforce deployment model:

1. Understanding how to merge and deploy code from one Salesforce Org to another.

2. The Metadata API doesn't support rolling-back once the API call has completed successfully.

3. Not all elements can be deployed using the Metadata API, because either Salesforce has not yet added those items to the Metadata API or they are configuration changes unique to each Org.

The Options

There are two options that Salesforce provides to deploy updates from one Salesforce Org to another:  Change Sets and the Force Migration Tool.

Change Sets

Change Sets are a UI-driven deployment function, which works well in Salesforce.  The creator simply selects each of the items that have changed, packages them into a Change Set and determines the target Salesforce Org(s).  Change Sets are probably the most-used process to deploy Salesforce updates, but include the following concerns:

• The changes must exist in the Salesforce Org where the Change Set is being created.

• Once the Change Set is created and deployed, it cannot be updated.  A new (or cloned) copy must be created.

• The Change Set creator must know every item that changed when building the Change Set.  

• A Change Set is limited in size and cannot include all design elements in most Salesforce implementations.

• When a Salesforce implementation contains several design elements and updates, the process to continue to use the "More-based" pagination system can become quite tedious.

• There really isn't a source control/change history aspect to this delivery method.

Force Migration Tool

The Force Migration Tool is an Ant-based mechanism to deploy changes based upon a set of source files.  This approach uses Apache Ant and a custom Salesforce connector to talk to the Metadata API directly.  As a result, the Force Migration Tool yields the following benefits:

• Change sets do not need to exist in a given Salesforce Org and are sourced from an external repository.

• Using source control, changes can be made to the source files just as if they were used in a Java or .NET application.

• There is no need to know each and every change, since the source control system is able to identify these updates.

• An entire Salesforce implementation can be pushed to another Org using the Force Migration Tool.

• Using the Force Migration Tool, changes can be removed using an un-deploy, destructive change process.

The Force Migration Tool is not 100% challenge-free, as the following issues have been found:

• Use of regular expressions within the build.xml are required to remove elements at run-time which exist in Salesforce but do not seem to migrate from one Org to the next.

• Use of Ant-based commands (like delete) is often required to remove files that cannot be migrated.

• The process requires continuous attention, especially during Salesforce core releases, to address new challenges.   New elements are often added or old items are removed, both of which can cause errors with the Force Migration Tool deployments.

The Solution

Having experience with the Continuous Improvement methodology, Eclipse and Apache Ant, I decided to leverage the Atlassian suite of tools (Stash/Git, Jira and Bamboo) for our Salesforce Continuous Integration.  At a high level, the flow that was created is listed below:

1. Developers use a Salesforce Org to make changes for their Jira issues.

2. The QA Team tests the changes in the same Salesforce Org.

3. Upon approval, the developer creates a new branch in Stash/Git and then uses Eclipse with the Force IDE to pull the changes from their Salesforce Org into their branch.  Once checked-in, Bamboo automatically fires all the tests built in Salesforce.

4. Once the Bamboo tests are green, the developer creates a Pull Request in Stash/Git, which is reviewed by the other team members.

5. Once the Pull Request is approved and merged, the Salesforce Org used for integration is used to perform a test deploy with the approved code and the tests are fired in that Org.  At this point, other developers will see this code when they create a new branch (as noted in step three).

6. If the merged code should be considered for a release candidate, a manual task is kicked off in Bamboo, which accomplishes the following tasks:

   1. A copy (or archive) of the source code is copied.

   2. The release candidate receives a build number.

   3. The Salesforce Org for integration is updated and all tests are executed.

7. At this point, the release candidate can be pushed to any other Salesforce Org.  This includes developer Orgs, the full copy Org (used for full testing), and Production.  With each deployment, all tests are fired as a precaution.  

Unfortunately, there are still items that cannot be migrated automatically.  To handle this aspect, a Production Support page has been added to our Atlassian Confluence application to document and track manual changes.  Fortunately for our team, the amount of items on this page pail in comparison to the amount of changes we are able to successfully deploy using the process noted above.

What's Next?

Since this was designed to be a high-level review of Continuous Integration with Salesforce, the next step is to perform a deeper dive into the following areas:

• Configuring Stash/Git for Salesforce with the Force IDE for Eclipse

• Configuring Bamboo for Salesforce Integration 

Look for these posts soon.

Have a really great day!