qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Helen Kwong <helenkw...@gmail.com>
Subject Re: Possible optimizations in Java client to reduce memory footprint
Date Tue, 07 May 2013 17:56:02 GMT
Hi Rob,

Thanks, I'll raise a JIRA for this with my changes then.

Helen


On Tue, May 7, 2013 at 12:18 AM, Rob Godfrey <rob.j.godfrey@gmail.com>wrote:

> Hi Helen,
>
> thanks for taking the time to investigate this. Your suggestions sound very
> sensible, and it would be good to get them reviewed and incorporated as
> soon as possible.  Would it be possible for you to raise a JIRA [1] for
> this and attach your changes as a patch file? - that will be the quickest
> way for us to be able to get these improvements applied.
>
> Many thanks for your efforts,
> Rob
>
>
>
> [1] https://issues.apache.org/jira/browse/QPID
>
>
> On 7 May 2013 03:10, Helen Kwong <helenkwong@gmail.com> wrote:
>
> > Hello,
> >
> > My team is using the Java broker and Java client, version 0.16, and
> looking
> > to lower the client's memory footprint on our servers. We did some heap
> > analysis and found that the consumption is coming mostly from
> > AMQAnyDestination instances, each having a retained size of 2864B, and
> > since we have 6000 queues on each of our 2 brokers, this amounts to about
> > ~33MB, which is valuable real estate for us. In our analysis we found a
> few
> > possible optimizations in the Qpid code that would reduce the per-queue
> > heap consumption and which don't seem high risk, and were wondering how
> > feasible these changes are.
> >
> > The first optimization involves cacheing / resuing redundant maps, while
> > the rest involves making the default value of a variable the constant
> empty
> > Map or List, instead of a new empty Map or List.
> >
> >
> > 1. In Address / AddressParser, cacheing / reusing the options Maps for
> > queues created with the same options string. (This optimization gives us
> > the most significant savings, but has the highest risk.)
> >
> > All our queues are created with the same options string, which means each
> > corresponding AMQDestination has an Address that has an _options Map that
> > is the same for all queues, i.e., 12K copies of the same map. As far as
> we
> > can tell, the _options map is effectively immutable, i.e., there is no
> code
> > path by which an Address’s _options map can be modified. (Is this
> correct?)
> > So a possible improvement is that in
> > org.apache.qpid.messaging.util.AddressParser, we cache the options map
> for
> > each options string that we've already encountered, and if the options
> > string passed in has already been seen, we use the stored options map for
> > that Address. This way, for queues having the same options, their Address
> > options will reference the same Map. (For our queues, each Address
> _options
> > Map takes up 1384 B, so with 12K queues we'd save about 15.5MB.)
> >
> >
> > 2. AMQDestination's _link field --
> > org.apache.qpid.client.messaging.address.Link$Subscription
> >
> > Subscription's args field is by default a new HashMap with default
> capacity
> > 16. In our use case it remains empty for all queues. A possible
> > optimization is to set the default value as Collections.emptyMap()
> instead.
> > As far was we can tell, Subscription.getArgs() is not used to get the map
> > and then modify it. For us this saves 128B per queue.
> >
> >
> > 3. AMQDestination's _targetNode field --
> > org.apache.qpid.client.messaging.address.Node$ExchangeNode
> >
> > Optimization A: The field _exchangeOptions is never used and can be
> > removed. It's an empty new HashMap with default capacity 16. For us this
> > saves 128B per queue.
> >
> > Optimization B: The parent class Node has a _bindings List that is by
> > default a new ArrayList with the default capacity of 10. In our use case
> > _bindings remains empty for all queues. A possible optimization is to set
> > the default value as Collections.emptyList() instead, and if
> addBindings()
> > is called, lazily set it as a new empty list first. For us this saves 80B
> > per queue.
> >
> >
> > 4. AMQDestination's _sourceNode field --
> > org.apache.qpid.client.messaging.address.Node$QueueNode
> >
> > Same 2 optimizations as (3) (with the unused field for QueueNode being
> > _queueOptions).
> >
> >
> > Overall: For us, the 5 optimizations in (2), (3), and (4) would save us
> > 128B x 3 + 80B x 2 = 544B per queue, and (1) would save us 1384B per
> queue;
> > all together this would save 1928B out of 2864B, or 67%, per queue for
> us.
> >
> > We'd appreciate feedback on how feasible these changes are and whether we
> > are making any incorrect assumptions. Thanks a lot!
> >
> > Helen
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message