qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Greig" <robert.j.gr...@gmail.com>
Subject Re: Java M3 Qpid broker memory consumption
Date Mon, 03 Nov 2008 20:41:01 GMT
2008/11/3 Keith Chow <keith.chow@xml-asia.org>:

> Broker heap size: 8gb. jmap shows most of the heap were consumed by byte
> arrays

OK - looking down the full histogram, it looks like there are 191000
messages in the broker. Is that what you expect based on the queue

> Running jmap's heapdump through a profiler showed MemoryMessageStore and
> InMemoryMessageHandle were holding huge map of messages (3GB+ and 1GB+).
> This result was somewhat expected since we have not yet configured any max
> message age, max queue depth, etc. and some of the clients were slower than
> the rest. But the strange thing is we think we also saw high memory usage
> with mostly empty queues but forgot to take a heap dump for analysis.

This is key - if you think the memory use is due to a bug that is very
different from slow consumer issues.

> Is there an easy way to set global limit for all queues by broker config?

I am not sure - can someone confirm? I know there is a threshold but
that used to emit only a warning to the log.

The issue here is a default that can be configured for temporary
queues bound to the topic exchange (I assume)? Ideally I think we want
to be able to specify that consumers get disconnected when certain
thresholds are hit (e.g. number of messages or size of messages).

Can someone from the C++ side indicate whether the C++ broker does
this? If not I shall raise enhancement requests for both brokers.

> What is the recommend way to deal with slow or fluctuating client using qpid
> broker/client?

You could set message TTL.

Given the size of your heap, you must have either very large messages
or publishers that are significantly faster than the consumers?


View raw message