Q&A With Claus Ibsen on Apache Camel

If you have ever been faced with the task of integrating multiple enterprise services, Apache Camel is definately a tool you should have in your toolbox. In this Q&A, Claus Ibsen shares some insights into the origins of Camel, how to get started, some of the challenges working on a such an ambitious project, and where Camel is going.

What was the inspiration for the development of Camel?

A decade ago the book Enterprise Integration Patterns was published. The book contains a catalog of integration solutions that its authors, Gregor Hohpe and Bobby Wolff, had encountered during their years as integration consultants. You can say this book is the equivalent of the famous gang-of-four-book.

James Strachan, the creator of Apache Camel, meet Gregor in person to discuss his book, and after that he was inspired to write a software implementation of the book. At the same time he was involved with other integration projects such as Apache ActiveMQ and ServiceMix, and the desire to have a reusable routing library that could run in both of them, and as well outside was another factor.

So in essence he wanted to write a small independent routing library that implements the EIP patterns from the book. And in summer 2007 the first release of Apache Camel was ready.

What has the community reaction been like to Camel?

The reaction has been great. Apache Camel together with Spring Integration are by far the most popular integration libraries. Apache Camel is used in other sister Apache projects such as Apache ActiveMQ and ServiceMix, and commercial products.

The community has grown year over year, and the fact we have so many Camel components today is because the community has built custom components and then donated those to Apache. We have also seen an uptake on 3rd party tooling that hooks into Camel, around monitoring, visualization and developer tooling. We'd love to see more of that, and its great to see what people can do with Camel.

James Strachan has for a long time wanted a web console which can visualize your running Camel applications, and because of that need, he created the hawtio project (http://hawt.io/). In this project we have a Camel plugin that can make your Camel application come to life. We see a lot of interest in this project form the Camel community. But hawtio is much more than a Camel plugin, and as such has its own community and home at github.

Over the years as Apache Camel become more and more popular and have been replacing commecial vendors expensive and heavy-weight integration platforms, a growing need for commercial support and consultancy for Apache Camel became apparent. And in this space we have several companies that offers their services around Camel, as well smaller business as well. This is a great testimony to the community, that Apache Camel is not only purist free open source project, but there is `real serious business` as well. So you can start on the free version and then acquire the commercial support as well.

What are some of the biggest challenges integrating so many technologies with one project?

At Apache Camel we have a lot of Camel components (adapters) to all kind of 3rd party transports, protocols and systems. We have about 200 give or take. That is a lot to maintain and integrate with, and as such the code base is large. The reason we can manage to keep up with that is because of our large community where we have many users who contribute to the project. And because of the big variety of the various Camel components we have people with different sorts of expertise and interest helping our with what they are passionate about.

To help support all these components we have more than 16k unit tests, and as such there is work to ensure they are all running and passing on the various platforms and JDKs. On top of that we do get a fair amount of PR and patches that require time to review and work together with the author before they can be accepted into the codebase.

But it's not as much the source code that is the biggest challenge, it's more the documentation that goes with it. Because the Apache Camel projects was created so many years ago, it wasn't the norm to provide documentation from the source, so the website and documentation is using a wiki system. And therefore it's a separate procedure to keep the documentation up to date. But we are working on changing all that. We have worked over the last year or so on letting the Camel components be self documented. That means the developers should document the components using a mix of standard javadoc, and a readme.md file. Then the build system is able to grab the documentation from the source, and auto update parts of the readme.md file, and as well generate documentation as html, pdf, epub and so on. That documentation is even also provided in the components so you at runtime can access that, for example from tooling. This allows us to present many more details about your running Camel application in 3rd party tooling and monitoring systems. This is an area that puts Camel ahead of others in this space.

What advice would you give developers looking to use Camel for the first time?

The Apache Camel website does not do a good job of hand-helding the developer to tell them what Apache Camel is, and how to get started using it. In fact what we have seen is that its often the best guides have been developed by other Camel end users. And as such we have a link collection on the Camel website that list a lot of those. However again it may be a struggle to find the best ones among those. So here is my advice on good links to look at:

This article captures best IMHO the concepts and idea of Apache Camel.

https://dzone.com/articles/open-source-integration-apache

For an opinionated and different pitches what Apache Camel is then this SO question is a good source:

http://stackoverflow.com/questions/8845186/what-exactly-is-apache-camel

Then I suggest to download Apache Camel, and take a look at the examples directory. A good beginner example is the camel-example-console.

We are aware of the Camel website and documentation needs an overhaul, and its a major goal for Camel 3.0, to provide a new and improved website, with better getting started experience.

For the users who want to dive more into Apache Camel and want a book like experience, then the free chapter 1 of our book Camel in Action is a recommended read (notice a 2nd edition of the book is currently in writing).

http://manning.com/ibsen/chapter1sample.pdf

What other projects besides Apache Camel are you working on?

I work on Java cloud technology in the fabric8 project (http://fabric8.io/) and also have fun hacking on non Java stuff such as the web console project called hawtio (http://hawt.io/). I am heavily involved with these projects, with a Camel focus, as both of them provides Camel related tooling and features. For example hawtio has a visual web console to manage and see your Camel applications at runtime. This is a fun aspect to take all the data Camel exposes and try to present this in a graphical way.

On the fabric8 project which is based on Kubernetes and Docker, then its all the new hot stuff, that is a great way to keep you up what is happening. My focus in this project is to bring in the 'Java developer' piece of the puzzle. For example the Java developer needs a set of tools so its easy to build your Java projects as Docker images. And as well for deploying into Kubernetes, to generate the kubernetes deployment descriptors.

I am also working on Camel tooling using JBoss Forge, that allows us to build a set of Camel commands, that are reusable in CLI, Eclipse, NetBeans, IDEA and on the web/rest. This is really exiting, as before the Camel tooling was heavy-weight and often exclusive for Eclipse. With JBoss Forge we now have all that tooling available to a much wider audience. For example you will be able to edit your Camel endpoints using a properties wizard, that present all those options in a type-safe UI. The tooling is able to do this for both XML and Java code, which is also something new. Before most of the Camel tooling are only able to edit Camel routes in XML.

What are your favourite blog posts, tutorials, videos or books on the subject of Camel?

At the Apache Camel website we have an excessive link collection to all sorts of Camel related material

http://camel.apache.org/articles.html

As a leading Camel developer I do fairly often blog about Apache Camel, so I suggest to keep an eye on my blog (http://www.davsclaus.com/). For a good beginner Camel video then I think the gr8conf video from 2014 has the best video and sound quality (http://www.youtube.com/watch?v=_pEwoztbfYI&feature=youtu.be&a), and if you have longer time then the videos from Barcelona are longer and has more Camel stuff.

If you are interested in new technologies like docker, kubernetes, and what we do in the fabric8 project, then I recommend to take at look at our medium blog site: https://medium.com/fabric8-io.

My favorite blog piece of late is from James Strachan on his thoughts on the decline of monolith application servers, and the rise of docker containers. As James says - Docker changed everything: How we develop, deploy and run applications: https://medium.com/fabric8-io/the-decline-of-java-application-servers-when-using-docker-containers-edbe032e1f30#.fvn952vaw

A real awesome video is James Rawlings demo of fabric8 devops capabilities, how to setup a full automated CI/CD pipeline with a single click on a Kubernetes platform: https://medium.com/fabric8-io/automation-for-microservices-5a8ed0ea100d#.eiygkx8ej

For Camel books then myself and Jonathan wrote the Camel in Action book, and we are currently wrting the 2nd edition (http://manning.com/ibsen2). I enjoyed reading Evgeniy's short book which was like introducing you to Apache Camel based on an use-case that was evolving. Jakub's and Scott's book with the 100 recipies is also a great book (http://www.packtpub.com/apache-camel-developers-cookbook/book). But I have to say that if you want all the details about Apache Camel, then the Camel in Action book is the best. For the 2nd edition Manning has it for sale at 39% discount if you order with the coupon code: dzibsen2

And on that bombshell I would say goodbye, now go and buy my book.