activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Sitsky <s...@nuix.com>
Subject Re: Trimming pending message cursors - big performance improvement
Date Tue, 05 Feb 2008 22:33:51 GMT
While we are on this topic - are there any further optimisations that 
can be done?  What about messages which have already been acked?  I 
assume they have to be left in the list in case a transaction is 
rolled-back and the message has to be re-delivered, or is that not the 
case?  I haven't read enough of the code to find out.

In my application, I have some very long-running transactions, and it 
would be great to make the pending message cursor list as small as 
possible so that when dispatchPending() is called, it isn't performing a 
lot of redundant computation over and over again.

Cheers,
David

David Sitsky wrote:
> Hi,
> 
> In my application using the current trunk version, I was finding that my 
> queue subscriptions had huge pending message cursors sizes (> 70,000 and 
> growing) - essentially they were equal to enqueueCounter - 
> dispatchCounter which I could see from JMX.
> 
> It seems like a simple optimisation could be to remove a node from the 
> cursor if has been dropped when we call dispatchPending().  This made a 
> gigantic difference in speed, since the broker wasn't iterating over a 
> huge number of dropped messages, and locks were held for a shorter 
> period of time.  Not to mention, less memory and CPU will be consumed by 
> the broker.
> 
> Here is the patch below - I don't know if it is safe in general, but it 
> seemed to work fine for me.
> 
> Cheers,
> David
> 
> Index: 
> activemq-core/src/main/java/org/apache/activemq/broker/region/PrefetchSubscription.java

> 
> ===================================================================
> --- 
> activemq-core/src/main/java/org/apache/activemq/broker/region/PrefetchSubscription.java

> (revision 618650)
> +++ 
> activemq-core/src/main/java/org/apache/activemq/broker/region/PrefetchSubscription.java

> (working copy)
> @@ -453,7 +461,11 @@
>                              if (node == null) {
>                                  break;
>                              }
> -                            if (canDispatch(node)) {
> +                else if (((QueueMessageReference)node).isDropped())
> +                {
> +                pending.remove();
> +                }
> +                            else if (canDispatch(node)) {
>                                  pending.remove();
>                                  // Message may have been sitting in the 
> pending
>                                  // list a while waiting for the 
> consumer to ak the message.


Mime
View raw message