Migrating AnypointMQ-Based Mulesoft Service to Serverless World
In this article, we demonstrate how we migrated MuleSoft-based integration services that are interconnected via AnypointMQ-based messaging queue infrastructure into a completely serverless architecture. Let's start by looking at the existing integration architecture running on the MuleSoft platform.
Current Architecture
In our exercise, we have two MuleSoft services involved. The first service acts as a producer, and the second service acts as a consumer. The producer service accepts messages from mobile, web, and other internal enterprise services via a REST endpoint, and it places the payload into the AnypointMQ queue. The message is delivered to the queue after being transformed into a standard format (canonical) for use by all consumers in the enterprise. Additionally, the message undergoes a few standard validations before it is transformed.
The second service is a consumer service that retrieves the message from the AnypointMQ queue. The message is then transformed from the canonical format to CSV or pipe-separated values and routed to the appropriate FTP folders.
Serverless Architecture
When transitioning to a serverless architecture, it is essential to carefully choose the appropriate systems and components to replace the existing ones. In our case, we are substituting MuleSoft with AWS Lambda, AnypointMQ with Amazon SQS, and Anypoint Designer with Kumologica.
In this architecture, AWS Lambda will serve as the hosting platform, providing a perfect serverless compute unit for our services. Amazon SQS will seamlessly take the place of AnypointMQ, ensuring there is no impact on the message delivery mechanism while leveraging the benefits of a serverless messaging infrastructure. Exposing APIs to the outside world will be accomplished through AWS API Gateway.
For development purposes, Kumologica Designer, a low-code development tool similar to MuleSoft, will be employed. This tool simplifies the migration of Mule flows to Kumologica flows, ensuring a smooth transition and ease of development for our services.
Building the Service
We will be using Kumologica Designer to build the services. Kumologica flows are based on NodeJS; hence, we need to have NodeJS installed on our machine. First, we will be building the producer service, which drops the message into the queue.
Producer Service
Producer service in MuleSoft is exposed using APIKit router and is an HTTP-based endpoint that accepts the request payload in JSON format. The accepted JSON format is transformed to CSV format or tab-separated format using Dataweave. Based on one of the attributes in the payload, the choice router in the flow will decide which FTP connector it needs to send the final file. Now, let's see how we replaced this MuleSoft flow with Kumologica nodes.
Note: Dataweave processors in MuleSoft use dataweave expression, whereas Datamapper uses JSONata expression. JSONata expressions are very similar to dataweave with few differences in syntax and functions. Both of them follow function chaining.
The Kumologica flow will look as shown below:
Consumer Service
In consumer service, Anypoint Listener picks the message from Anypoint MQ. Based on the target field attribute in the message using the choice router, the flow decides whether to transform as CSV payload or tab-separated payload. The message is transformed using Dataweave. The transformed message is then passed to the FTP connector. Now, let's see how we replaced this MuleSoft flow with Kumologica nodes.
The Kumologica flow will look as shown below:
