Error handling and monitoring (Advanced)

Integrating disparate applications through asynchronous messaging increases the possibility of failures, and makes error handling a mandatory part of every integration application. Camel offers a couple of mechanisms for handling and recovering from error conditions. In this tutorial, we will use Dead Letter Channel that retries failing requests and if the error remains, moves the failing message to a Dead Letter Queue (DLQ).

Getting ready

The complete source code for this tutorial is located under the project camel-message-routing-examples/error-handling.

How to do it...

An error handler can either be global, applying to all routes in a CamelContext, or applied to individual routes. There are different types of error handlers, each with different options and behaving slightly differently. In our example, we will create a DeadLetterChannel error handler applied to one route only.

1. We start with the errorHandler declaration, give it a DeadLetterChannel type, and configure its redeliveryPolicy:

   <errorHandler id="deadLetterErrorHandler" type="DeadLetterChannel" deadLetterUri="mock:error" useOriginalMessage="true">
       <redeliveryPolicy maximumRedeliveries="3" redeliveryDelay="1000"
                         backOffMultiplier="2" useExponentialBackOff="true"/>
   </errorHandler>

2. Then, we apply the error handler to the route by its ID:

   <route errorHandlerRef="deadLetterErrorHandler">
       <from uri="direct:start"/>
       <transform>
           <simple>${in.body} Modified data!</simple>
       </transform>
       <to uri="mock:result"/>
   </route>

How it works...

As we already know, the consumer is responsible for receiving messages from other systems, creating an Exchange and then it starts routing it. If an error occurs before the routing of a message, for example, if the consumer cannot read a file from the file system, error handling will not be triggered. It is the responsibility of the consumer to deal with errors arising before the start of the routing process. Camel error handling applies only during routing of Exchanges. When a Processor (Endpoint or EIP) throws an exception during routing, Camel catches the exception and stores it in the Exchange's exception field. The error handler kicks in only when the Exchange contains a caught exception. There are different error handlers available and one is always selected:

• DefaultErrorHandler: This is used by default if no other error handler is configured. It behaves similarly to a Java error handling mechanism. It doesn't perform any retries and propagates the exception back to the caller.

• LoggingErrorHandler: This logs the message along with the exception.

• NoErrorHandler: Camel always needs to have an error handler, using this dummy handler makes it behave as if there isn't any.

• TransactionErrorHandler: This is the default error handler used in transacted routes and requires a transaction manager to rollback the transactions for failed messages.

• DeadLetterChannel: This is the one used in our example. This error handler is an implementation of a Dead Letter Channel pattern where if a message cannot be delivered to its designated target, the message is moved to a different channel, which is quite often called a dead letter queue.

In our example, for the mandatory deadLetterUri option, we have specified mock:error, so if any error happens during the routing, the error handler will take the control, cleanup the exception from the Exchange, and move the message to this endpoint. Notice that deadLetterUri can also be another route with multiple steps, where we could still access the exception that caused the failure from an Exchange property:

Exception exception = exchange.getProperty(Exchange.EXCEPTION_CAUGHT, Exception.class);

The important thing to remember is that the exception field of the Exchange will be cleaned up and for the caller it looks like the Exchange has been processed successfully.

Usually in a route, we have multiple steps (processors), which are connected by channels, and each step can modify the message. When an exception is thrown from a processor, the channel responsible for that processor notices the exception and passes the control to the error handler. Depending on where the exception has occurred, it is possible that the Exchange has been modified from successfully completed processors and DeadLetterChannel will get a modified Exchange. To solve this problem, errorHandler has the useOriginalMessage option. When this option is set, DeadLetterChannel will receive an Exchange containing the original In Message Body that was passed to the route. So, in our example, if the mock:result throws an exception, even though the message body has been modified by the transform step, the DeadLetterChannel's endpoint mock:error will receive the original message passed to the route. This option is useful for situations where we want to replay the failed Exchanges again to the same route using the original message.

Certain kinds of errors, for example caused by database deadlock, network failure, and so on are likely to succeed when retried a couple of times a little later. For these kinds of scenarios, DeadLetterChannel(and also DefaultErrorHandler and TransactionErrorHandler) offers a redelivery feature which is configured with a RediliveryPolicy. For our example, we have allowed up to three retry attempts with a 1 second delay between the attempts. In addition, we have set it to exponentially back off and use backOffMultiplier of 2, so the second attempt will be done 2 seconds after the first one, and the third and final, attempt will be done 4 seconds after the second. If all of these attempts to process the request fail, only then the message will be moved to DLQ.

There's more...

There are two other mechanisms in Camel for dealing with erroneous situations: Exception Clause and doTry-doCatch-doFinally construct. The first one can be used together with error handlers, whereas the latter allows catching exceptions similarly to the Java language. Also, we will have quick look at monitoring and logging tools used with Camel.

Exception Clause

The Exception Clause allows us to specify error handling logic per exception type(s) in a route or the whole CamelContext scope. It is very flexible as we can have multiple Exception Clauses at any scope with one or multiple matching exceptions in each. Let's have a look at an example that has a route scope Exception Clauses applying for two Exception types:

<route>
    <from uri="direct:start"/>
    <onException>
        <exception>org.camel.ValidationException</exception>
        <exception>org.camel.OrderFailedException</exception>
        <redeliveryPolicy maximumRedeliveries="1"/>
        <handled>
            <constant>true</constant>
        </handled>
        <to uri="mock:error"/>
    </onException>
    <process ref="orderValidator"/>
    <to uri="mock:result"/>
</route>

When an error is thrown in a route, Camel will try to find the best matching Exception Clauses by going through all clauses from first to last (top-down) and then each declared exception type and comparing it to the thrown exception root cause. The comparison is done by starting from the root cause of the exception hierarchy, a top-down exception declaration compared to a bottom-up exception hierarchy. If an exact match is not found, Camel will try to find the closest matching exception using the instanceof operator. This comparison will match the Exception Clauses that has an exception type which is the closest superclass of the thrown Exception.

Once a matching Exception Clause is found, Camel will apply its redeliveryPolicy (if there is one) and other options such as handled, continued, and useOriginalBody before routing the message (if there are endpoints specified).

Java style error handling

Camel has the doTry-doCatch-doFinally construct for error handling which works in a similar way to Java's try-catch-finally blocks. It is not as flexible as error handlers or the Exception Clause but allows applying try-catch-finally logic in a route.

<route>
    <from uri="direct:start"/>
    <doTry>
        <process ref="orderValidator"/>
        <doCatch> 
<exception>org.camel.ValidationException</exception>
<exception>org.camel.OrderFailedException</exception>
            <to uri="mock:catch"/>
        </doCatch>
        <doFinally>
            <to uri="mock:finally"/>
        </doFinally>
    </doTry>
</route>

Keep in mind that when doTry-doCatch-doFinally is used, the regular error handler and Exception Clause will not apply.

Monitoring and logging

In asynchronous message oriented applications, monitoring and logging play a more significant role. A graph of routes, where a message can move in any direction depending on rules evaluated at runtime, is not easy to troubleshoot. Luckily, Camel has various tools to help developers write messaging applications which are easy to monitor and debug. Here are a few of them:

• JMX Support (http://camel.apache.org/camel-jmx.html): Camel has extensive JMX support and allows monitoring and control of Camel managed objects through a JXM client. By default, the JMX instrumentation agent is enabled, and Camel will register the managed objects with MbeanServer. Supported managed types include CamelContext, Routes, Endpoints, Components, Processors, ErrorHandlers, and so on. We can connect to locally running CamelContext with a JMX client such as JConsole, see some performance statistics, and even interact with the objects. If the Camel process doesn't appear on the local connections list of the JMX client, we can connect to Camel by first running it with the RMI connector server enabled:

  -Dorg.apache.camel.jmx.createRmiConnector=True

  After enabling the RMI connector server connecting to it as a remote process using the following URL:

  service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi/camel

• Hawtio: This is a new hot management console for Java applications. It is a third party application with a plugin for Camel, and provides a web interface for visualizing and interacting with Camel routes. Worth trying it http://hawt.io/.

• Log Component (http://camel.apache.org/log.html): We have used logging in the examples so far to log each Exchange. But, log component can also run as a throughput logger where it aggregates statistics about Exchanges and logs them periodically or once a certain number of Exchanges are aggregated. The following endpoint will log stats for every 10 messages:

  .to("log://org.apache.camel.howto?level=DEBUG&groupSize=10")

  Whereas, this one will log every 10 seconds only if there is at least one Exchange:

  .to("log://org.apache.camel.howto?groupInterval=10000&groupActiveOnly=true")

  Log component uses SL4J which supports Mapped Diagnostic Contexts (MDC). MDC is a technique used for stamping each log entry with contextual information for easier debugging and auditing complex distributed multithread applications. To get it working, we have to use a test kit that supports MDC (such as Log4j or logback) and enable it in Camel:

  camelContext.setUseMDCLogging(true);

  Then, in our log configuration (such as lo4j.properties), we can specify contextual information such as exchangeId, messageId, routeId, camelContextId, and so on:

  log4j.appender.out.layout.ConversionPattern=%d [%-15.15t] %-5p %-30.30c{1} - %-10.10X{camel.exchangeId} - %-10.10X{camel.routeId} - %m%n

• Tracer (http://camel.apache.org/tracer.html): This is an InterceptStrategy which allows detailed tracing of routes. It logs how an Exchange moves from one endpoint to another during routing. By default, it is disabled, but we can enable it by running the Camel application with -t or trace arguments or by setting it programmatically to CamelContext:

  camelContext.setTracing(true);