Transforming messages (Intermediate)

Very often integration applications have to work with existing systems which have a predefined data format that we cannot change. In other situations, after some processing the data has to be converted to a format that can be understood by external systems. Camel offers many different ways for transforming data from one format into another. In this tutorial we are going to convert XML input into JSON using the xmljson component and will have a look at other ways for doing data transformations.

Getting ready

The complete source code for this tutorial is located under the project camel-message-routing-examples/transforming-messages.

In addition to the standard camel-core dependencies, for this tutorial we will need also the xmljson component:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-xmljson</artifactId>
    <version>${camel-version}</version>
</dependency>

How to do it...

1. Within the dataFormat element configure an xmljson data format and give it an ID.

   <dataFormats>
       <xmljson id="xmljsonWithOptions" forceTopLevelObject="true" trimSpaces="true" skipNamespaces="true"
   removeNamespacePrefixes="true"/>
   </dataFormats>

2. Then, add a marshal step in the route and reference the xmljson data format by its ID.

   <route>
       <from uri="direct:start"/>
       <marshal ref="xmljsonWithOptions"/>
       <to uri="mock:result"/>
   </route>

How it works...

Camel data formats are used to transform data between a low-level presentation and a high-level presentation by providing two operations: marshall and unmarshall. XmlJson data format is marshaling from XML to JSON and unmarshaling from a JSON to XML format. There are other data formats that convert from Java to XML and JSON or vice versa, but all of them require a POJO. The difference for this data format is that it doesn't require a POJO for the conversion. The only thing we did was to instantiate a dataFormat instance with the xmljson type and configure its options for this specific transformation.

When the default options are good enough, there is no need to instantiate a data format instance. Instead, as part of the route, we only specify the operation type (marshall/unmarshall) and the data Format type:

<marshal><xmljson/></marshal>

There's more...

Message translator is a very general pattern and data transformations happen in many different places in Camel. We will see now different techniques for transforming messages and a very useful pattern for normalizing messages into a common format.

Type conversion

Type conversions happen very often in pipeline applications where a message is passed through multiple steps. Compared to data formats, these are simpler conversions, for example from File to InputStream, from String to byte[], and so on. Camel handles these conversions transparently by maintaining an internal registry of available converters, and using them whenever needed. For example, whenever a message reaches a Processor that expects the payload to be from a different type, Camel will try to convert that message payload into the expected type using the type converters. If there are no type converters in the TypeConverterRegistry to convert the message from its current format to the expected format, then it will throw an exception and the processing will fail. Another occasion, when type conversion happens is when the message body is explicitly converted to a type as part of the routing:

from(...)
    .convertBodyTo(String.class)
    .to(...)

Or when the message body is requested in a specific type:

Document document = message.getBody(Document.class);

Transforming with expression language

The transform command followed by an expression is a quick and easy way to modify the message. Inside the transform command we can use any of the supported languages and access all fields of the Exchange: properties, headers, and in and out bodies. For example, using Simple language it is possible to do quick alterations to the message, or do full transformations using languages such as JQuery, XSLT, Groovy, Ruby, and so on.

</transform>
    <simple>New message ${body}</simple>
</transform>

Executing a Java method

When none of the existing solutions are good enough for the intended transformation, it is possible to do the job manually by executing a Java bean method. Depending on our needs, it is possible to annotate the method parameters and get different parts of the message, such as properties, headers, or body as method arguments:

.to("bean:myConverter?method=convert(${body}, ${header.userId})")

Instead of calling a custom bean, another option is to implement the Processor interface and call it as part of the route. The advantage of this approach is that we will get as the method parameter the actual Exchange object so we have full control over what transformation to do, and we can implement it inline as part of the route:

from("...")
    .process(new Processor() {
        public void process(Exchange exchange) throws Exception {
            exchange.getIn().setBody("Changed body");
        }
    })
    .to("...");

Using template component

There are Camel components which act literally as message translators. These are template components that convert the message into another format using a template file. Such components are XSLT, Velocity, FreeMarker, and Scalate.

Normalizing messages to a common format

In some occasions, semantically equivalent data is received in different formats from disparate sources and it has to be converted to a common data format for uniform processing throughout the system. In these situations using the Normalizer pattern can be a good way to go, as shown in the following diagram:

As we can see in the preceding diagram, the Normalizer uses a Context-Based Router to route each message to an appropriate message translator depending on the message format. Once the messages are transformed to a common format they all can be processed in a unified way. We can see example routes using this pattern in Spring XML or Java DSL at http://camel.apache.org/normalizer.html.