camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Newsham <jnews...@referentia.com>
Subject jms concurrentConsumers is not competing consumers
Date Sat, 04 Feb 2012 02:55:59 GMT

We are using Camel 2.9.0.  We are using the "concurrentConsumers=n" 
option on our jms consumer endpoint so that we can handle numerous InOut 
requests concurrently.  In Camel documentation, this option is described 
as supporting competing consumers, which I interpret to mean a pool of 
threads which are _competing_ to handle incoming requests.  I expect 
that whenever a request comes in, that request will be processed by some 
idle thread from the thread pool (if there are any).  However, this is 
not that behavior we are observing.  Instead, I observe some requests 
not being processed immediately (or for quite some time), despite having 
numerous idle consumer threads, and this is causing a severe liveness 
issue for our application.

Let me describe a simple scenario.  Assume we have a single InOut jms 
queue with concurrentConsumers=3, and the following stream of messages 
(and required processing time):

Message #0 - 1h
Message #1 - 1s
Message #2 - 1s
Message #3 - 1s
Message #4 - 1s
Message #5 - 1s
Message #6 - 1s
Message #7 - 1s
Message #8 - 1s
Message #9 - 1s
Message #10 - 1s

Because Message #0 requires 1 hour to process, it will occupy one 
concurrent consumer thread for 1 hour.  Nevertheless, we expect the 
remaining 2 threads should be able to handle the remaining 9 messages 
very quickly.  Instead, messages #3, #6, #9 are not processed until 
Message #0 completes processing -- an hour later.  Based on this 
observation, it appears that messages are dispatched to queues owned by 
individual threads, in round-robin fashion, instead of having threads 
compete to consume messages from a single queue.  When some messages can 
take a long time to process, it can mean that messages can queue up and 
go unprocessed for a very long time, even when there are free threads.

As you can see, this is not "competing consumers".  Is the above 
behavior considered a bug?  Are there any suggested workarounds?

Thanks!
Jim

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message