From dev-return-9491-apmail-activemq-dev-archive=activemq.apache.org@activemq.apache.org Wed Feb 06 00:16:49 2008 Return-Path: Delivered-To: apmail-activemq-dev-archive@www.apache.org Received: (qmail 19072 invoked from network); 6 Feb 2008 00:16:49 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 6 Feb 2008 00:16:49 -0000 Received: (qmail 70097 invoked by uid 500); 6 Feb 2008 00:16:41 -0000 Delivered-To: apmail-activemq-dev-archive@activemq.apache.org Received: (qmail 70065 invoked by uid 500); 6 Feb 2008 00:16:41 -0000 Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list dev@activemq.apache.org Received: (qmail 70056 invoked by uid 99); 6 Feb 2008 00:16:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Feb 2008 16:16:41 -0800 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rajdavies@gmail.com designates 209.85.162.183 as permitted sender) Received: from [209.85.162.183] (HELO el-out-1112.google.com) (209.85.162.183) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Feb 2008 00:16:12 +0000 Received: by el-out-1112.google.com with SMTP id j27so906132elf.0 for ; Tue, 05 Feb 2008 16:16:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; bh=rhSVslN9TzJngyTCX+THLLl8XN/ZjHPYjvk4rI0HX+0=; b=ml7vQy/xTpOtChfftbLCVY09SzMw26UdF8tIPGW6HPHbDSyGmpRVKlzdH02F+VO1Fwx3OBuVOuuy/kzOMyebTNcTFpgIKOCTmtZ1TEc4iuRK0ABgBK6J1bbb/9tsvmPsBnW8saJ3rlBdVt+v9iUy0Iwypp/hduFPi3EaKUQ8IB4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=TMqgi/qmJc5k3FEIWKdfXdwEP0PWPhewmTijTOANRwlluFZnVfoey9woTuXfhDcLEcei63nnSwC01iSmF2KKVSdD2SO5v7Bcmw9+qwXsufN0AQ20q1G/O2+HZRWdhnesXuZZ/8i0rfkAXgKfSWfGyDcc0neVOzYuTQ0dwILlIdQ= Received: by 10.150.137.8 with SMTP id k8mr3859539ybd.61.1202256979343; Tue, 05 Feb 2008 16:16:19 -0800 (PST) Received: from ?192.168.1.244? ( [86.134.103.250]) by mx.google.com with ESMTPS id f7sm489301nfh.39.2008.02.05.16.16.17 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 05 Feb 2008 16:16:18 -0800 (PST) Message-Id: <55221644-5A92-41E8-B914-132927CEA40C@gmail.com> From: Rob Davies To: dev@activemq.apache.org In-Reply-To: <47A8D506.1080503@nuix.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: Trimming pending message cursors - big performance improvement Date: Wed, 6 Feb 2008 00:16:15 +0000 References: <47A8D506.1080503@nuix.com> X-Mailer: Apple Mail (2.915) X-Virus-Checked: Checked by ClamAV on apache.org Hi David, I'll give it a whirl! cheers, Rob On Feb 5, 2008, at 9:28 PM, David Sitsky wrote: > Hi, > > In my application using the current trunk version, I was finding > that my queue subscriptions had huge pending message cursors sizes > (> 70,000 and growing) - essentially they were equal to > enqueueCounter - dispatchCounter which I could see from JMX. > > It seems like a simple optimisation could be to remove a node from > the cursor if has been dropped when we call dispatchPending(). This > made a gigantic difference in speed, since the broker wasn't > iterating over a huge number of dropped messages, and locks were > held for a shorter period of time. Not to mention, less memory and > CPU will be consumed by the broker. > > Here is the patch below - I don't know if it is safe in general, but > it seemed to work fine for me. > > Cheers, > David > > Index: activemq-core/src/main/java/org/apache/activemq/broker/region/ > PrefetchSubscription.java > =================================================================== > --- activemq-core/src/main/java/org/apache/activemq/broker/region/ > PrefetchSubscription.java (revision 618650) > +++ activemq-core/src/main/java/org/apache/activemq/broker/region/ > PrefetchSubscription.java (working copy) > @@ -453,7 +461,11 @@ > if (node == null) { > break; > } > - if (canDispatch(node)) { > + else if (((QueueMessageReference)node).isDropped()) > + { > + pending.remove(); > + } > + else if (canDispatch(node)) { > pending.remove(); > // Message may have been sitting in > the pending > // list a while waiting for the > consumer to ak the message.