qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Chow" <keith.c...@xml-asia.org>
Subject Re: Java M3 Qpid broker memory consumption
Date Sat, 08 Nov 2008 04:15:55 GMT
We applied the server side queue limits, server/client lowered prefetech
high/low marks, and simplified our test case as follows:

1) One fast producer at 200 msg/s, msg size ~250bytes, non-transactional,
non-persistent, no-ack mode, TTL = 5s.
2) 2 to 3 extremely slow and/or suspended consumers subscribed to the same
topic.

We also modified broker's expire message task to remove any node with
expired = true, ignoring durable / acquire condition (to make sure they're
purged from the message store).

Result:

Broker size old generation heap would still reach gigabytes in size in less
than 10 minutes. JConsole showed no significant messages had built up in any
queues much higher the prefetech count.

Profiling showed the gigabytes of byte[] were referenced by the broker's
pool event Job queue. And the events themselves where referenced by
underlying mina's SocketImpl.

The cause is similar to this TCP congestion issue from the apache mina users
list, http://mina.markmail.org/message/6q5t5gwdozypm6dk?q=byte%5B%5D+gc

Is this expected behaviour with M3 java broker with slow client?

As an interim solution, we've modified the broker to detect slow topic
consumers (by inspecting expiry timestamp for our usecase) and kill them off
(with mina protocol's close session call). This allowed
GC to reclaim the dead client's memory resource.

Keith

On Thu, Nov 6, 2008 at 5:14 AM, Gordon Sim <gsim@redhat.com> wrote:

> Robert Greig wrote:
>
>> 2008/11/4 Gordon Sim <gsim@redhat.com>:
>>
>>  Can someone from the C++ side indicate whether the C++ broker does
>>>> this? If not I shall raise enhancement requests for both brokers.
>>>>
>>> The c++ broker allows a hard limit to be set for a queue or a system wide
>>> default.
>>>
>>
>> What actions can you configure when the limit is hit? It occurs to me
>> that there are two main cases:
>>
>> 1) "regular" queue - in this case you want to limit the publisher
>>
>> 2) private temporary queue bound to a topic exchange (or indeed other
>> exchange types) - in this case you probably want to kill the slow
>> consumer
>>
>> Thoughts?
>>
>
> I agree.
>
> At present the c++ broker is much more limited and can either: kill the
> publisher, flow to disk, or discard the oldest message(s) to make room.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message