Apache Camel addresses need for discrete infrastructure for services mediation and routing

Read a full transcript of the discussion. Listen to the podcast. Sponsor: IONA Technologies.

One size fits all has its limits. Most developers prefer to cherry-pick their infrastructure resources, keep them lightweight, and remain as agile as they can.

Taking a clue from this philosophy, the Apache Software Foundation has dozens of open source projects under way to build out the discrete infrastructure assets through community input and involvement, while also providing a needed balance between choice and automation.

One such project, Apache Camel, a sub-set of Apache ActiveMQ, is nearing maturity milestones that will make it a unique approach to middleware services mediation and routing. It's also becoming an essential ingredient to IONA's FUSE offerings.

To learn more about Apache Camel and its value to developers and operators, I recently sat down with James Strachan, technical director of engineering at IONA Technologies, and longtime Apache developer and committer.

Here are some excerpts from the discussion:

The problem we’re trying to address is the routing and mediation problem, which lots of people have. They're taking data from various components and sources -- whether it’s files, databases, message queues, Web services, instant messaging, or other data systems, integrating them together, formatting them, and connecting them to the systems. From a higher level perspective, this could be for legacy integration of systems, for smart routing, performance, monitoring, or testing or monitoring transaction flows.

Apache Camel grew organically from code and ideas from a bunch of other Apache projects, particularly Apache ActiveMQ and Apache ServiceMix. We found that people wanted to create and use patterns from the "Enterprise Integration Patterns" book in many different scenarios.

I definitely recommend people read Gregor Hohpe's book "Enterprise Integration Patterns." He offers a really good patterns catalog of how people should do mediation and integration. Rather like the original Gang of Four "Design Patterns" book, which describes low-level programming things, Gregor’s book describes very well how enterprise integration patterns (EIPs) can work and gives us a language for describing them.

Some people wanted to use these patterns inside an Enterprise Service Bus (ESB), some people wanted to use these patterns inside a message broker, and other people wanted to use these patterns inside an application itself or to talk between messaging providers. Still other people wanted to use them inside a Web services framework or some other communication platform.

What we tried to do with Camel was give it the smallest footprint possible, so that it can be reused anywhere, whether in a servlet, in the Web services stack, inside a full ESB, or a messaging application.

We work very closely with the ServiceMix community at Apache which has created a complete Java Business Integration (JBI)-compliant container of JBI components. Camel can be deployed within the ServiceMix ESB among JBI components, but some people don’t use JBI and they may just use Java Message Service (JMS) or they may just use Web services or they may just be JAX-WS clients or whatever. So, we try to make Camel agnostic to technologies. You can use it within patterns like Spring, or JBI or OSGi, or you can use it within any application.

Across all of IT we’re seeing increased specialization in many different areas, where the specialization helps us solve a problem at a higher level. ... And because we’re doing it an open-source environment where there’s a community involved, there's more likelihood that this will be applied across many other different types of platforms and technology.

Camel is unique in a number of ways. What we’re doing with Camel is defining a high-level language to describe EIPs, which I don’t think anybody else has done before. The other thing that’s unique is that this language very closely maps to components that work inside Spring 2. ... What we’re doing is raising the abstraction level to make things very simple, reducing the amount of XML we have to write, but still exposing the wire-level access if you need to do the really hard stuff and roll your sleeves up and get down and dirty.
People don’t have to worry about the low-level details of how to use JMS, how to use JBI, or how to wire together the Spring components correctly, and so forth. We're giving people a nice, simple, high-level abstraction, but yet we are exposing all the power of frameworks like Spring and still exposing the low-level details if you need them.

People really want small and simple-to-use components that solve the problems they have. I've seen that throughout all of our customers. People ask for very specific solutions. They don’t say, "Give me a SOA." They say, "I need a message router," "I need a message bus," "I need an ESB," or "I need a services framework." Often, people have very specific requirements and are very much cherry-picking the best-of-breed components from the open-source tool set

If you go to http://open.iona.com, there is a whole raft of documentation online forums, a wiki, and so forth.

Read a full transcript of the discussion. Listen to the podcast. Sponsor: IONA Technologies. Produced by Interarbor Solutions: analysis, consulting and rich new-media content production.