camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Jiang <willem.ji...@gmail.com>
Subject Re: async send with preserved order
Date Thu, 02 Jun 2011 03:22:19 GMT
Hi Jim,

I think you may consider to use the resequencer on your FIFO endpoint 
side. If you just want to speed up the sending processor.

[1]http://camel.apache.org/resequencer.html

Willem

On 6/2/11 5:22 AM, Jim Newsham wrote:
>
> Hi Claus,
>
> Yeah I mentioned the single-threaded executor in my initial email, as
> the only way I know to make that work, and also pointed out the
> undesirable performance impact this approach has. I would really like
> for all messages to be sent out in order without waiting for the result
> of each message before sending the next message. For example:
>
> Send 1; send 2; send 3; send 4; send 5.
> Wait for response for 1, 2, 3, 4, 5 (using async callback).
> [parallel/fast]
>
> Instead of:
>
> Send 1; wait for response for 1.
> Send 2; wait for response for 2.
> etc.
> [serial/slow]
>
> My uninformed assessment is that this is not possible in Camel, since
> sending need to be synchronous/serial and receiving responses needs to
> be asynchronous yet these two operations form a single method call --
> processor.process(). Does that sound right? If so, this is more of an
> idealistic wish list item for camel 3 or camel 4. :)
>
> Regards,
> Jim
>
>
> On 6/1/2011 6:40 AM, Claus Ibsen wrote:
>> Are you sure you want preserved order on the sender side?
>> In essence to do that you may have to use a thread pool of 1, so the
>> tasks are processed in sequence.
>> I would assume the JDK in fact does grab the taks by order by default
>> as the thread pool executor uses a queue for the tasks (FIFO).
>>
>> On the other hand if you want to process the responses in the same
>> order as the tasks was submitted, then you can use a CompletionService
>> for that.
>>
>> There is no out of the box in the JDK, so we created one in Camel as
>> we needed it. See the org.apache.camel.util.concurrent package.
>>
>>
>>
>> On Wed, Jun 1, 2011 at 7:04 AM, Jim Newsham<jnewsham@referentia.com>
>> wrote:
>>> Is it possible to send asynchronously while preserving order? For
>>> example:
>>>
>>> for (int i = 0; i<= COUNT; i++) {
>>> Object body = Integer.valueOf(i);
>>> producerTemplate.asyncCallbackRequestBody(URL, body, callback);
>>> }
>>>
>>> Assuming that "URL" is a FIFO queue, is there some way for the
>>> recipient to
>>> receive the messages in the order that the sender has sent them? I think
>>> the answer is no, since async calls just schedule a task on an
>>> executor, and
>>> it's possible for the thread scheduler to choose/process send tasks
>>> out of
>>> order. [*]
>>>
>>> If that's correct, then *if order is important*, all sends must be
>>> synchronous, and performance will suffer (transmission will be in serial
>>> instead of in parallel, and any network latency will make matters worse
>>> since InOut requires round-trip communication). For this case, it really
>>> would be useful to be able to async send and guarantee transmission
>>> order
>>> (for JMS or whatever endpoint being used).
>>>
>>> [*] One could set a single-threaded executor on the producer
>>> template. The
>>> call would be asynchronous and transmission in order, however the actual
>>> send operations would still be in serial (each InOut message must be
>>> completely processed before the next message is sent), so the
>>> performance
>>> would still suffer.
>>>
>>> Comments?
>>> Jim
>>>
>>>
>>>
>>>
>>
>>
>
>


-- 
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog:    http://willemjiang.blogspot.com (English)
          http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang

Mime
View raw message