Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <47A8E44F.1010402@nuix.com>
Date: Tue, 05 Feb 2008 17:33:51 -0500
From: David Sitsky <sits@nuix.com>
User-Agent: Thunderbird 2.0.0.9 (Windows/20071031)
MIME-Version: 1.0
To: dev@activemq.apache.org
Subject: Re: Trimming pending message cursors - big performance improvement
References: <47A8D506.1080503@nuix.com>
In-Reply-To: <47A8D506.1080503@nuix.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

While we are on this topic - are there any further optimisations that 
can be done?  What about messages which have already been acked?  I 
assume they have to be left in the list in case a transaction is 
rolled-back and the message has to be re-delivered, or is that not the 
case?  I haven't read enough of the code to find out.

In my application, I have some very long-running transactions, and it 
would be great to make the pending message cursor list as small as 
possible so that when dispatchPending() is called, it isn't performing a 
lot of redundant computation over and over again.

Cheers,
David

David Sitsky wrote:
> Hi,
> 
> In my application using the current trunk version, I was finding that my 
> queue subscriptions had huge pending message cursors sizes (> 70,000 and 
> growing) - essentially they were equal to enqueueCounter - 
> dispatchCounter which I could see from JMX.
> 
> It seems like a simple optimisation could be to remove a node from the 
> cursor if has been dropped when we call dispatchPending().  This made a 
> gigantic difference in speed, since the broker wasn't iterating over a 
> huge number of dropped messages, and locks were held for a shorter 
> period of time.  Not to mention, less memory and CPU will be consumed by 
> the broker.
> 
> Here is the patch below - I don't know if it is safe in general, but it 
> seemed to work fine for me.
> 
> Cheers,
> David
> 
> Index: 
> activemq-core/src/main/java/org/apache/activemq/broker/region/PrefetchSubscription.java 
> 
> ===================================================================
> --- 
> activemq-core/src/main/java/org/apache/activemq/broker/region/PrefetchSubscription.java 
> (revision 618650)
> +++ 
> activemq-core/src/main/java/org/apache/activemq/broker/region/PrefetchSubscription.java 
> (working copy)
> @@ -453,7 +461,11 @@
>                              if (node == null) {
>                                  break;
>                              }
> -                            if (canDispatch(node)) {
> +                else if (((QueueMessageReference)node).isDropped())
> +                {
> +                pending.remove();
> +                }
> +                            else if (canDispatch(node)) {
>                                  pending.remove();
>                                  // Message may have been sitting in the 
> pending
>                                  // list a while waiting for the 
> consumer to ak the message.