qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fraser Adams <fraser.ad...@blueyonder.co.uk>
Subject Qpid JMS setJMSReplyTo memory consumption.
Date Sun, 14 Oct 2012 17:59:07 GMT
Hello,
I'm currently implementing a QMF2 REST API and whilst working on the 
mechanism to invoke QMF2 methods I started messing around on some soak 
tests where I did a POST specifying a given QMF2 Object, which then does 
an invokeMethod on the QmfConsoleData on the REST Server.

What I noticed was that the memory was growing, so for a couple of weeks 
I was tearing my hair out thinking I had a bug somewhere in my code. 
Eventually I narrowed things down to a call to 
"request.setJMSReplyTo(destination);" in the invokeMethod() call in Console.

I then wrote a little cut down test to really hammer it, the guts of it 
are as follows:

             Connection connection = 
ConnectionHelper.createConnection(url, "{reconnect: true}");
             _console = new Console(this);
             _console.addConnection(connection);

             // First we create a large number of queues using the QMF2 
create method on the broker object
             List<QmfConsoleData> brokers = 
_console.getObjects("org.apache.qpid.broker", "broker");
             if (brokers.isEmpty())
             {
                 System.out.println("No broker QmfConsoleData returned");
                 System.exit(1);
             }

             QmfConsoleData broker = brokers.get(0);
             QmfData arguments = new QmfData();

             while (true)
             {
                 try
                 {
                     MethodResult results = 
broker.invokeMethod("getLogLevel", arguments);
                 }
                 catch (QmfException e)
                 { // This may be throw if we've already added the 
queues, we just catch and ignore for this test.
                     //System.out.println(e.getMessage());
                 }
             }

So it's just calling the getLogLevel method on the broker 
ManagementObject within a tight loop

I've attached a graph from jconsole showing the memory consumption over 
time.

As you can see it's not quite a leak rather it eventually settles down, 
though when the consumption "converges" CPU utilisation climbs to ~100% 
so for high rate request/response calls this could be an issue.

I've tracked this behaviour down to the use of 
"java.lang.ref.SoftReference" in AMQMessageDelegate_0_10.java in 
qpid.client.message


Although this isn't truly a memory leak it does seem to be less than 
ideal behaviour, in my case I've actually only got a couple of 
Destination objects but setJMSReplyTo is creating SoftReference 
instances on each call, which frankly seems excessive for most 
request/response use-cases as these SoftReferences stick around for 
quite a while.

I'd have thought that something like a proper LRU cache would have been 
more efficient (and less memory hungry), indeed something simple such as 
having a custom class extending TimerTask to wrap the Destination and 
ReplyTo so that when the TimerTask run() is invoked it can expire itself 
if it hasn't been used. As most request/response use-cases involve quite 
a high temporal correlation between the request and response a 
relatively modest expire time of say a few minutes would be sufficient 
and indeed if the time between request and response is high it's fairly 
likely that the cache isn't going to cause any significant performance 
gain anyway.

Thoughts??

BTW This was observed in qpid 0.12 so I guess things may have changed in 
later versions? 0.8 didn't use a SoftReference it looks like it was 
using a custom ReferenceMap

Regards,
Frase










Mime
View raw message