Event Meshes and Multi-Cloud
Digital transformation is leading to the accelerated adoption of cloud-native applications. These applications typically use a microservices-style architecture that span across multiple zones to provide for scalability. Most cloud-native applications even span across multiple cloud providers in order to complete complex business processes such as banking, logistics, and telehealth. Such a complex business process requires orchestration among multiple microservices (sometimes in hundreds), multi zones, multi-cloud providers, and multiple data centers.
The event-driven architecture (EDA) style is central to orchestrated execution of business processes that are compartmentalized by business contexts and implemented through microservices. The EDA also plays a key role around data in the cloud-native space, which is bounded to microservices, and will need to be managed as different segments of transactions are processed across multiple microservices.
Event 101
An event is a change or action in an enterprise system that produces a message. This message is then published in an event sourcing system in an enterprise. This message may be consumed by other systems in the enterprise which might trigger further action.
For example, consider this transaction: a bank account holder is trying to transfer money to a foreign account. When this transaction is initiated, it constitutes an event that produces a message about this transaction by such-and-such user is being initiated. The fraud and risk systems in the bank that consume these messages might trigger action to check if the transaction to a foreign bank is within regulatory compliance limits and/or if the transaction is fraudulent.
There are several tools that support event- and messaging-based architecture. However, Apache Kafka, the open-source software, is the most popular event/messaging system. Some of its differentiators include its ability to horizontally scale, high throughput, and ability to process high-volume events with very low latency. Also, Apache Kafka implements the messaging stream architecture with Advanced Message Queuing Protocol (AMQP) with features like transaction management, clustering of APIs, etc. In addition, Red Hat has made an enterprise-grade event streaming software (AMQ Streams) with an optimized Apache Kafka solution directly on the OpenShift container platform. OpenShift with Knative eventing for Kubernetes makes it a highly desirable container platform for a solid implementation of microservices-based applications on containers with end to end.
In essence, in a large enterprise, hundreds of thousands of events/messages are published and consumed while enterprise business processes are orchestrated. When such a large amount of events are produced and consumed, particularly serving applications across multiple business domains, zones, data centers, and cloud providers, how do you manage such a large stream of events?
Enter Event Mesh!
An event mesh is a dynamic infrastructure that propagates events across a broad range of producers and consumers within the enterprise boundary (on-premises data centers and cloud providers) as well as the partner ecosystem.
Such a mesh infrastructure is needed to transport, monitor, manage and operate very high volumes of events flowing through heterogeneous application topology with multiple application language run times (Java, C++, Python, etc.), using multiple application integration/interoperation protocols (JMS, FTP, HTTP, SOAP, RPC, etc.) spread over multiple cloud environments (IBM, AWS, Azure, GCP, etc.). Such infrastructure also enables the basis of separation of concerns with respect to interoperability among the portfolio of applications so the app developers focus on the basic functions and business logic of the applications.
The combination of an EDA and event mesh establishes a simplified environment with regard to application development leaving the app developers to only be concerned about the core applications.
Conclusion
EDA is a central and key technology strategy for large enterprises adopting cloud-native technologies, particularly with hybrid/multi-cloud ecosystems. Horizontally scalable and reliable event messaging systems (such as Apache Kafka and Red Hat AMQ Streams that support Knative event patterns) help you design a highly distributed application topology. When such an event/messaging system is distributed across multiple locations and cloud providers, an event mesh is a required event management infrastructure. It comes with a key set of features that support multiple interaction patterns and reduces complexity for developers of modern applications that implement near real-time, loosely coupled design patterns.
Additional Reference and Video Presentation:
Why Event Meshes Should Be On Your IoT Radar
How to Create a Global Multi-Cloud Event Mesh [Video]
