camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pham Ngoc Hai <pngoc...@yahoo.com>
Subject Re: jms concurrentConsumers is not competing consumers
Date Sat, 04 Feb 2012 04:32:35 GMT
Hi Jim,
Since it's InOut queue, the reply is processed by a single thread as stated in here: http://camel.apache.org/jms.html
no matter how many concurrentConsumer you set.
Regards,
Ngoc Hai.






________________________________
 From: Jim Newsham <jnewsham@referentia.com>
To: users@camel.apache.org 
Sent: Saturday, February 4, 2012 10:55 AM
Subject: jms concurrentConsumers is not competing consumers
 

We are using Camel 2.9.0.  We are using the "concurrentConsumers=n" option on our jms consumer
endpoint so that we can handle numerous InOut requests concurrently.  In Camel documentation,
this option is described as supporting competing consumers, which I interpret to mean a pool
of threads which are _competing_ to handle incoming requests.  I expect that whenever a request
comes in, that request will be processed by some idle thread from the thread pool (if there
are any).  However, this is not that behavior we are observing.  Instead, I observe some
requests not being processed immediately (or for quite some time), despite having numerous
idle consumer threads, and this is causing a severe liveness issue for our application.

Let me describe a simple scenario.  Assume we have a single InOut jms queue with concurrentConsumers=3,
and the following stream of messages (and required processing time):

Message #0 - 1h
Message #1 - 1s
Message #2 - 1s
Message #3 - 1s
Message #4 - 1s
Message #5 - 1s
Message #6 - 1s
Message #7 - 1s
Message #8 - 1s
Message #9 - 1s
Message #10 - 1s

Because Message #0 requires 1 hour to process, it will occupy one concurrent consumer thread
for 1 hour.  Nevertheless, we expect the remaining 2 threads should be able to handle the
remaining 9 messages very quickly.  Instead, messages #3, #6, #9 are not processed until
Message #0 completes processing -- an hour later.  Based on this observation, it appears
that messages are dispatched to queues owned by individual threads, in round-robin fashion,
instead of having threads compete to consume messages from a single queue.  When some messages
can take a long time to process, it can mean that messages can queue up and go unprocessed
for a very long time, even when there are free threads.

As you can see, this is not "competing consumers".  Is the above behavior considered a bug? 
Are there any suggested workarounds?

Thanks!
Jim
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message