camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Krasser <>
Subject Re: AW: Reliable resequencing
Date Tue, 30 Sep 2008 07:48:54 GMT

Hi Dirk,

sorry for the late reply. I opened a JIRA ticket for reliable resequencing
( I'll try to address
your questions (how to handle delivery failures ... etc.) when working on a
patch. Did you make any progress in the meantime (ideas, patches ...)?


Dirk Mahler wrote:
> Hi Martin,
> sorry for the late response - lot's of work is waiting ;-) But
> nevertheless thanx for your answer even if it contains bad news for us...
> I'd like to give you some feedback on your ideas - please note that I do
> not know Camel and it's internal structures very well.
> Martin Krasser wrote:
>> - Batch resequencer: read n messages within a single transaction (instead
>> of
>> n transactions as done currently in 1.4), reorder them, send them to a
>> destination and commit the transaction. This can easily done within a
>> single
>> thread.
> Might work but I doubt that the batch sequencer makes sense in a real
> world because IMHO it fails on the borders between two batches. A short
> example: Imagine a batch size of 3 and an incoming message order
> 1|2|4|3|5|6. This would result in sequential processing of two batches.
> There is no need to do any re-ordering in any of these batches but
> messages 3 and 4 will finally be delivered in wrong order to the consumer.
> Martin Krasser wrote:
>> - Stream resequencer: the situation is more tricky here because there is
>> a
>> continuous stream of messages without having candidates for transaction
>> boundaries (in contrast to the batch resequencer where the batch size can
>> be
>> used to determine a transaction boundary). I'm still not 100% sure how to
>> solve this problem but a possible solution might roughly look like this:
>> Read messages from a source endpoint with n threads, suspend them until
>> they
>> have been re-ordered based on the message they have read and
>> resume/execute
>> them sequentially after their message becomes eligible for delivery.
>> After
>> delivery of the message each thread commits the transaction it started.
>> Consequently, each message is delivered to a destination within a
>> separate
>> transaction (in contrast to the batch resequencer where several messages
>> can
>> be delivered within a single transaction) but the resequencing is
>> reliable
>> too.
> What I understand seems to be similar to the thoughts I have in mind about
> how it could work. I'll try to explain in my own words:
> MessageExchanges are consumed by the resequencer in n threads. Each thread
> is stopped until the predecessor of this message exchange has been
> delivered to the final consumer or the configured timeout is over. In the
> first case the MessageExchange is sent in the same thread it came from to
> the consumer. After receiving the response from there this will be sent
> back to the originating message producer.
> Or shorter: this is just synchronous communication which is stopped (->
> messageExchange.wait(timeout)) - until a required precondition is met ->
> messageExchange.notify(). This way there should be no problem of
> propagating a transaction context or an error state.
> One further question: what should happen, if delivery of a MessageExchange
> to the final consumer fails - should all pending MessageExchanges be
> rejected or delivered? Maybe several strategies should be provided.

View this message in context:
Sent from the Camel - Users mailing list archive at

View raw message