camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Newsham <jnews...@referentia.com>
Subject Re: jms concurrentConsumers is not competing consumers
Date Mon, 06 Feb 2012 19:54:26 GMT

Hi Ngoc Hai,

I'm not talking at all about the producer side where replies are 
processed, but the consumer side where requests are being received and 
processed.  Incoming requests are not being dispatched to idle threads; 
instead, they are backing up behind a single (non-idle) thread.

To me, this looks like a bug.  I am interested in hearing the opinions 
of the community on this point, and I'm willing to file a JIRA issue.

Thanks,
Jim

On 2/3/2012 6:32 PM, Pham Ngoc Hai wrote:
> Hi Jim,
> Since it's InOut queue, the reply is processed by a single thread as stated in here:
http://camel.apache.org/jms.html no matter how many concurrentConsumer you set.
> Regards,
> Ngoc Hai.
>
>
>
>
>
>
> ________________________________
>   From: Jim Newsham<jnewsham@referentia.com>
> To: users@camel.apache.org
> Sent: Saturday, February 4, 2012 10:55 AM
> Subject: jms concurrentConsumers is not competing consumers
>
>
> We are using Camel 2.9.0.  We are using the "concurrentConsumers=n" option on our jms
consumer endpoint so that we can handle numerous InOut requests concurrently.  In Camel documentation,
this option is described as supporting competing consumers, which I interpret to mean a pool
of threads which are _competing_ to handle incoming requests.  I expect that whenever a request
comes in, that request will be processed by some idle thread from the thread pool (if there
are any).  However, this is not that behavior we are observing.  Instead, I observe some requests
not being processed immediately (or for quite some time), despite having numerous idle consumer
threads, and this is causing a severe liveness issue for our application.
>
> Let me describe a simple scenario.  Assume we have a single InOut jms queue with concurrentConsumers=3,
and the following stream of messages (and required processing time):
>
> Message #0 - 1h
> Message #1 - 1s
> Message #2 - 1s
> Message #3 - 1s
> Message #4 - 1s
> Message #5 - 1s
> Message #6 - 1s
> Message #7 - 1s
> Message #8 - 1s
> Message #9 - 1s
> Message #10 - 1s
>
> Because Message #0 requires 1 hour to process, it will occupy one concurrent consumer
thread for 1 hour.  Nevertheless, we expect the remaining 2 threads should be able to handle
the remaining 9 messages very quickly.  Instead, messages #3, #6, #9 are not processed until
Message #0 completes processing -- an hour later.  Based on this observation, it appears that
messages are dispatched to queues owned by individual threads, in round-robin fashion, instead
of having threads compete to consume messages from a single queue.  When some messages can
take a long time to process, it can mean that messages can queue up and go unprocessed for
a very long time, even when there are free threads.
>
> As you can see, this is not "competing consumers".  Is the above behavior considered
a bug?  Are there any suggested workarounds?
>
> Thanks!
> Jim


Mime
View raw message