camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bach Christian" <Christian.B...@six-group.com>
Subject Lost Messages in DeadLetterChannel.
Date Fri, 04 Sep 2009 07:04:33 GMT
Heya Camel Riders

I stumbled upon a problem on having DeadLetterChannel doing the error
handling in an staged architecture (all in-VM, non-distributed). Our
batch processing application (uses Camel 2.0 and) is losing messages. 

It seems to only happen under peak loads, when SEDA queues become full
and start to throw IllegalStateExceptions on performing the add()s to
their internal BlockingQueues. Our producer recovers those exchanges and
yield()s, before re-sending the Exchanges to the SEDA endpoint.

I have put together two unit tests that reproducibly illustrate the
problem and the absence thereof. Both unit tests are similar expect for
the dead letter queue exception handling implementation used. Both have
the SEDA queues, facing stages, configured to size=1 in order to emulate
peak load semantics.

The first test case (DeadLetterChannelTest) is configured to use Camel's
DeadLetterChannel and reproducibly loses messages, depending on the run
between 5 to 15 (out of 45) on my machine.

The second test case (ExceptionBarrierTest) is configured to use a
custom ExceptionBarrier endpoint in place of DeadLetterChannel. The
ExceptionBarrierComponent is derived from a Camel DirectComponent with
some custom code to handle Exchanges suffering from exceptions and
forwarding the erroneous exchanges to the dead letter queue. With the
ExceptionBarrier, which takes in its simplistic implementation advantage
of our specific routes to some extent, no messages are lost. 

The unit test are attached and I'd be most interested in learning what
is going wrong.

Cheers, Christian.


 <<camel-lost-messages.zip>> 
 
This message is for the named person's use only. It may contain confidential, proprietary
or legally privileged information. If you receive this message in error, please notify the
sender urgently and then immediately delete the message and any copies of it from your system.
Please also immediately destroy any hardcopies of the message. 
The sender's company reserves the right to monitor all e-mail communications through their
networks.

Mime
View raw message