activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Sitsky <s...@nuix.com>
Subject Re: Queue performance from recent changes
Date Mon, 10 Mar 2008 06:03:09 GMT
James Strachan wrote:
> On 06/03/2008, David Sitsky <sits@nuix.com> wrote:
>> I am sure it will be application-dependent, so making it a policy makes
>>  a lot of sense.  For my application, I only have a pending size of 1
>>  since each work item's processing requirements can vary tremendously.
> 
> I wonder could the same code be smart enough to work in the 2
> different modes based on the prefetch size?
> 
> i.e. use the default if the consumers's prefetch size is > 100 or
> something or use David's approach if its smaller
> 
> If not then using destination policies sounds fine to me; just
> wondered if we could be smart enough to use the right policy based on
> the consumer configuration?

I think it is very much application-dependent - and it is based on more 
than just the prefetch size.

In my situation, I may have 500,000 messages in my system that need to 
be delivered, but I don't want them to be delivered to pending queues 
unnecessarily, since it may be some time before the consumers have a 
chance to eat them up.  I also need a large queue page size since I 
can't do a commit() after each message received.  So I also have a lot 
of uncommitted messages floating about the system - maybe 24,000 at a 
given time.

I really need the requirement for only putting a message to a consumer's 
pending queue when it can process it.  Otherwise I found the pending 
queues for each consumer would grow to be extremely large, consuming 
unnecessary CPU and memory resources.  With my changes, the broker's 
usage was kept nice and small.

A lot of this may only occur for applications that have a large queue 
page size.

It feels right for this to be a policy option... I know how complex 
different messaging application performance requirements can be!

Cheers,
David








-- 
Cheers,
David

Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia    Ph: +61 2 9280 0699
Web: http://www.nuix.com                            Fax: +61 2 9212 6902

Mime
View raw message