Mailing-List: contact camel-user-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: camel-user@activemq.apache.org
Received-SPF: pass (athena.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <19737595.post@talk.nabble.com>
Date: Tue, 30 Sep 2008 00:48:54 -0700 (PDT)
From: Martin Krasser <dev@martin-krasser.de>
To: camel-user@activemq.apache.org
Subject: Re: AW: Reliable resequencing
In-Reply-To: <19396212.post@talk.nabble.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <19331003.post@talk.nabble.com>
 <000801c90ff8$ba0b2b50$17b2a8c0@goesser> <19396212.post@talk.nabble.com>


Hi Dirk,

sorry for the late reply. I opened a JIRA ticket for reliable resequencing
(https://issues.apache.org/activemq/browse/CAMEL-949). I'll try to address
your questions (how to handle delivery failures ... etc.) when working on a
patch. Did you make any progress in the meantime (ideas, patches ...)?

Cheers,
Martin


Dirk Mahler wrote:
> 
> Hi Martin,
> 
> sorry for the late response - lot's of work is waiting ;-) But
> nevertheless thanx for your answer even if it contains bad news for us...
> 
> I'd like to give you some feedback on your ideas - please note that I do
> not know Camel and it's internal structures very well.
> 
> 
> Martin Krasser wrote:
>> 
>> - Batch resequencer: read n messages within a single transaction (instead
>> of
>> n transactions as done currently in 1.4), reorder them, send them to a
>> destination and commit the transaction. This can easily done within a
>> single
>> thread.
>> 
> 
> Might work but I doubt that the batch sequencer makes sense in a real
> world because IMHO it fails on the borders between two batches. A short
> example: Imagine a batch size of 3 and an incoming message order
> 1|2|4|3|5|6. This would result in sequential processing of two batches.
> There is no need to do any re-ordering in any of these batches but
> messages 3 and 4 will finally be delivered in wrong order to the consumer.
> 
> 
> Martin Krasser wrote:
>> 
>> - Stream resequencer: the situation is more tricky here because there is
>> a
>> continuous stream of messages without having candidates for transaction
>> boundaries (in contrast to the batch resequencer where the batch size can
>> be
>> used to determine a transaction boundary). I'm still not 100% sure how to
>> solve this problem but a possible solution might roughly look like this:
>> Read messages from a source endpoint with n threads, suspend them until
>> they
>> have been re-ordered based on the message they have read and
>> resume/execute
>> them sequentially after their message becomes eligible for delivery.
>> After
>> delivery of the message each thread commits the transaction it started.
>> Consequently, each message is delivered to a destination within a
>> separate
>> transaction (in contrast to the batch resequencer where several messages
>> can
>> be delivered within a single transaction) but the resequencing is
>> reliable
>> too.
>> 
> 
> What I understand seems to be similar to the thoughts I have in mind about
> how it could work. I'll try to explain in my own words:
> 
> MessageExchanges are consumed by the resequencer in n threads. Each thread
> is stopped until the predecessor of this message exchange has been
> delivered to the final consumer or the configured timeout is over. In the
> first case the MessageExchange is sent in the same thread it came from to
> the consumer. After receiving the response from there this will be sent
> back to the originating message producer.
> 
> Or shorter: this is just synchronous communication which is stopped (->
> messageExchange.wait(timeout)) - until a required precondition is met ->
> messageExchange.notify(). This way there should be no problem of
> propagating a transaction context or an error state.
> 
> One further question: what should happen, if delivery of a MessageExchange
> to the final consumer fails - should all pending MessageExchanges be
> rejected or delivered? Maybe several strategies should be provided.
> 

-- 
View this message in context: http://www.nabble.com/Reliable-resequencing-tp19331003s22882p19737595.html
Sent from the Camel - Users mailing list archive at Nabble.com.