Integration: API or ESB?

Application integration is essential for any successful digital transformation initiative. From Gartner:

 “Application integration is the process of enabling independently designed applications to work together.” 

Application integration existed even before SOA or ESBs. In the 1990s, enterprises experienced the beginning of packaged solutions such as ERP and home-grown enterprise applications. In the real world, an application can’t live in isolation and has to exchange information with other enterprise applications over the network for successful completion of enterprise business processes. 

The Need for Middleware

An application typically contains code purpose-built to perform a certain business function. Let's call this code "business logic." As discussed earlier, this application has to integrate with other applications in an enterprise. This requires some connectivity related code within the application. Let's call these lines of code "connectivity logic." The connectivity code slowly grows within the application as we require more and more integration with other disparate applications. Sometimes, along with connectivity code, you might need some code for the transformation of data before delivering it to the target application. With time, the application becomes bulkier with connectivity code and slows down impacting its maintenance, longevity, and performance. This also gives rise to a point-to-point integration mess where the applications now are tightly coupled with each other. To avoid this pain, the data transformation logic and most of the connectivity logic should live in an external software component known as middleware.

You may also enjoy: The Complexities of Continuous Integration, Continuous Delivery, and Sprint Planning

Rise of the ESB Pattern

ESB stands for Enterprise Service Bus. It is an architecture pattern that enables disparate applications to connect seamlessly with each other. It’s a kind of middleware that has capabilities such as data transformation (such as XML to JSON), protocol transformation (like FTP to HTTP), content-based message routing and service orchestration. Many vendors converted this concept into an ESB product with standard connectors. ESBs gained importance with the popularity of SOA. ESBs' ability to compose multiple services as a single service or exposing data from legacy system as a service helped enterprises achieve SOA with less code and minimal effort. The key benefits of using an ESB-based tool are faster time to market, re-usability, and agility. One could expose these services written in ESB as a web-service (SOAP or REST) or listen to a MOM (message-oriented middleware). MOM also plays a key role in an event-driven architecture along with the ESB.

API Connectivity

HTTP-based APIs (application programming interface) in this decade have gained a lot of popularity with the need for connectivity between digital technologies such as IoT, social, cloud, and mobile with the enterprise. In today’s world, there is a need to always stay connected meaningfully with your customers, partners, and suppliers to be ahead in the field. This requires a solid integration strategy and a hybrid integration platform capable of delivering that strategy. API-based connectivity is a key ingredient in a hybrid integration solution.

The creation of an API consists of a definition and an implementation.

API Definition 

This is where an API is described. The description of an API includes its operations allowed, message formats, authentication methods, endpoints, terms of use, and more. A thorough design is required to come up with a good scalable API description. There are two popular types of HTTP APIs, namely, REST and SOAP. The REST API is described using specifications such as OpenAPI or RAML. 

SOAP-based APIs are described using WSDL. It is essential to define your API in a manner by keeping its target consumers in mind. The API should be treated like a product. There are API management tools in the market that allow you to not just define an API but also manage the life cycle of an API including the versioning and setting of rate limit. Some vendors also provide some amount of lightweight ESB capabilities within the API management tool. 

API Implementation

This is where the actual logic of an API is written. API implementation is your back-end service invoked by the consumers with ways mentioned in the API definition. This back-end service could expose data from a packaged application such as an ERP, a CRM, or from a legacy application within the mainframe. This back-end service could be developed using any coding language such as Java, .NET, or just use an ESB.

API & ESB Together

Almost all vendors that sell an ESB product also sell an API management product, but the reverse may not be true; perhaps the reason is that API management is a natural extension to ESB to manage the API endpoints created using ESB. Almost all ESB vendors have evolved their ESB product into iPaaS to provide a cloud-based integration platform. Also, with the rise of microservices, vendors are modernizing their architecture of ESB product so it can be containerized. An enterprise that wants to thrive in today’s environment would need to have both API and ESB based integration capabilities to be a truly connected enterprise.