From dev-return-9778-apmail-activemq-dev-archive=activemq.apache.org@activemq.apache.org Mon Mar 10 06:03:31 2008 Return-Path: Delivered-To: apmail-activemq-dev-archive@www.apache.org Received: (qmail 72323 invoked from network); 10 Mar 2008 06:03:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 10 Mar 2008 06:03:30 -0000 Received: (qmail 94755 invoked by uid 500); 10 Mar 2008 06:03:27 -0000 Delivered-To: apmail-activemq-dev-archive@activemq.apache.org Received: (qmail 94728 invoked by uid 500); 10 Mar 2008 06:03:27 -0000 Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list dev@activemq.apache.org Received: (qmail 94719 invoked by uid 99); 10 Mar 2008 06:03:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Mar 2008 23:03:27 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [203.217.22.128] (HELO web1.nuix.com.au) (203.217.22.128) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Mar 2008 06:02:48 +0000 Received: from [192.168.222.61] (unknown [192.168.222.61]) by web1.nuix.com.au (Postfix) with ESMTP id BC5531DF101 for ; Mon, 10 Mar 2008 17:02:08 +1100 (EST) Message-ID: <47D4CF1D.7000403@nuix.com> Date: Mon, 10 Mar 2008 17:03:09 +1100 From: David Sitsky Organization: NUIX Pty Ltd User-Agent: Thunderbird 2.0.0.12 (Windows/20080213) MIME-Version: 1.0 To: dev@activemq.apache.org Subject: Re: Queue performance from recent changes References: <20080218094428.843041A9832@eris.apache.org> <47BA6480.4020604@nuix.com> <2C159E97-AD73-4BB1-879C-0505B6B6E801@gmail.com> <47BA9424.9040701@nuix.com> <93C79F5E-8F0C-4B65-B7A9-21687FD00CD0@gmail.com> <47BBC94B.8010302@nuix.com> <47CF8732.7020600@nuix.com> <0AA2DC4F-55F6-400B-BD40-FE17472C1A7B@gmail.com> <47D074BC.2010407@nuix.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org James Strachan wrote: > On 06/03/2008, David Sitsky wrote: >> I am sure it will be application-dependent, so making it a policy makes >> a lot of sense. For my application, I only have a pending size of 1 >> since each work item's processing requirements can vary tremendously. > > I wonder could the same code be smart enough to work in the 2 > different modes based on the prefetch size? > > i.e. use the default if the consumers's prefetch size is > 100 or > something or use David's approach if its smaller > > If not then using destination policies sounds fine to me; just > wondered if we could be smart enough to use the right policy based on > the consumer configuration? I think it is very much application-dependent - and it is based on more than just the prefetch size. In my situation, I may have 500,000 messages in my system that need to be delivered, but I don't want them to be delivered to pending queues unnecessarily, since it may be some time before the consumers have a chance to eat them up. I also need a large queue page size since I can't do a commit() after each message received. So I also have a lot of uncommitted messages floating about the system - maybe 24,000 at a given time. I really need the requirement for only putting a message to a consumer's pending queue when it can process it. Otherwise I found the pending queues for each consumer would grow to be extremely large, consuming unnecessary CPU and memory resources. With my changes, the broker's usage was kept nice and small. A lot of this may only occur for applications that have a large queue page size. It feels right for this to be a policy option... I know how complex different messaging application performance requirements can be! Cheers, David -- Cheers, David Nuix Pty Ltd Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699 Web: http://www.nuix.com Fax: +61 2 9212 6902