qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Greig" <robert.j.gr...@gmail.com>
Subject Re: Java M3 Qpid broker memory consumption
Date Mon, 03 Nov 2008 20:41:01 GMT
2008/11/3 Keith Chow <keith.chow@xml-asia.org>:

> Broker heap size: 8gb. jmap shows most of the heap were consumed by byte
> arrays

OK - looking down the full histogram, it looks like there are 191000
messages in the broker. Is that what you expect based on the queue
depths?

> Running jmap's heapdump through a profiler showed MemoryMessageStore and
> InMemoryMessageHandle were holding huge map of messages (3GB+ and 1GB+).
> This result was somewhat expected since we have not yet configured any max
> message age, max queue depth, etc. and some of the clients were slower than
> the rest. But the strange thing is we think we also saw high memory usage
> with mostly empty queues but forgot to take a heap dump for analysis.

This is key - if you think the memory use is due to a bug that is very
different from slow consumer issues.

> Is there an easy way to set global limit for all queues by broker config?

I am not sure - can someone confirm? I know there is a threshold but
that used to emit only a warning to the log.

The issue here is a default that can be configured for temporary
queues bound to the topic exchange (I assume)? Ideally I think we want
to be able to specify that consumers get disconnected when certain
thresholds are hit (e.g. number of messages or size of messages).

Can someone from the C++ side indicate whether the C++ broker does
this? If not I shall raise enhancement requests for both brokers.


> What is the recommend way to deal with slow or fluctuating client using qpid
> broker/client?

You could set message TTL.

Given the size of your heap, you must have either very large messages
or publishers that are significantly faster than the consumers?

Robert

Mime
View raw message