qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robbie Gemmell <robbie.gemm...@gmail.com>
Subject Re: Qpid JMS setJMSReplyTo memory consumption.
Date Sun, 14 Oct 2012 19:51:37 GMT
 Changes were made in this area via
https://issues.apache.org/jira/browse/QPID-3440 which puts them in 0.16 and
0.18 (or the 0.20 alpha due later in the week) if you want to give them a
try.

Robbie

On 14 October 2012 18:59, Fraser Adams <fraser.adams@blueyonder.co.uk>wrote:

> Hello,
> I'm currently implementing a QMF2 REST API and whilst working on the
> mechanism to invoke QMF2 methods I started messing around on some soak
> tests where I did a POST specifying a given QMF2 Object, which then does an
> invokeMethod on the QmfConsoleData on the REST Server.
>
> What I noticed was that the memory was growing, so for a couple of weeks I
> was tearing my hair out thinking I had a bug somewhere in my code.
> Eventually I narrowed things down to a call to "request.setJMSReplyTo(**destination);"
> in the invokeMethod() call in Console.
>
> I then wrote a little cut down test to really hammer it, the guts of it
> are as follows:
>
>             Connection connection = ConnectionHelper.**createConnection(url,
> "{reconnect: true}");
>             _console = new Console(this);
>             _console.addConnection(**connection);
>
>             // First we create a large number of queues using the QMF2
> create method on the broker object
>             List<QmfConsoleData> brokers = _console.getObjects("org.**apache.qpid.broker",
> "broker");
>             if (brokers.isEmpty())
>             {
>                 System.out.println("No broker QmfConsoleData returned");
>                 System.exit(1);
>             }
>
>             QmfConsoleData broker = brokers.get(0);
>             QmfData arguments = new QmfData();
>
>             while (true)
>             {
>                 try
>                 {
>                     MethodResult results = broker.invokeMethod("**getLogLevel",
> arguments);
>                 }
>                 catch (QmfException e)
>                 { // This may be throw if we've already added the queues,
> we just catch and ignore for this test.
>                     //System.out.println(e.**getMessage());
>                 }
>             }
>
> So it's just calling the getLogLevel method on the broker ManagementObject
> within a tight loop
>
> I've attached a graph from jconsole showing the memory consumption over
> time.
>
> As you can see it's not quite a leak rather it eventually settles down,
> though when the consumption "converges" CPU utilisation climbs to ~100% so
> for high rate request/response calls this could be an issue.
>
> I've tracked this behaviour down to the use of
> "java.lang.ref.SoftReference" in AMQMessageDelegate_0_10.java in
> qpid.client.message
>
>
> Although this isn't truly a memory leak it does seem to be less than ideal
> behaviour, in my case I've actually only got a couple of Destination
> objects but setJMSReplyTo is creating SoftReference instances on each call,
> which frankly seems excessive for most request/response use-cases as these
> SoftReferences stick around for quite a while.
>
> I'd have thought that something like a proper LRU cache would have been
> more efficient (and less memory hungry), indeed something simple such as
> having a custom class extending TimerTask to wrap the Destination and
> ReplyTo so that when the TimerTask run() is invoked it can expire itself if
> it hasn't been used. As most request/response use-cases involve quite a
> high temporal correlation between the request and response a relatively
> modest expire time of say a few minutes would be sufficient and indeed if
> the time between request and response is high it's fairly likely that the
> cache isn't going to cause any significant performance gain anyway.
>
> Thoughts??
>
> BTW This was observed in qpid 0.12 so I guess things may have changed in
> later versions? 0.8 didn't use a SoftReference it looks like it was using a
> custom ReferenceMap
>
> Regards,
> Frase
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message