activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Blair <rick.bl...@boeing.com>
Subject Re: Broker not releasing memory
Date Wed, 03 Dec 2008 17:38:33 GMT
Hi Kirby,

We have seen exactly the same behaviour.  As a work around we set
optimizeAcknowdlege to false.  I did not run down the root cause, however.
 
-- 
Cheers!

Rick Blair


> From: "Frugia, Kirby A" <kirby.frugia@hp.com>
> Reply-To: <users@activemq.apache.org>
> Date: Wed, 3 Dec 2008 15:43:30 +0000
> To: "users@activemq.apache.org" <users@activemq.apache.org>
> Conversation: Broker not releasing memory
> Subject: RE: Broker not releasing memory
> 
> This is fairly easy to reproduce if you create one app which sends messages
> quickly and another which consumes the messages and sleeps before ack'ing.
> Once you see broker memory usage going up (through activemq-admin), kill the
> consumer application.  Then run activemq-admin again some time later and
> you'll see the same memorypercentusage.
> 
> Or, let the consumer run the broker out of memory and you'll see the producer
> get blocked.  Kill both apps and you'll never see memory go back down again
> and you'll never be able to send messages again.
> 
> Thanks,
> Kirby
> 
> -----Original Message-----
> From: Will van der Leij [mailto:will@stonethree.com]
> Sent: Wednesday, December 03, 2008 12:17 AM
> To: users@activemq.apache.org
> Subject: RE: Broker not releasing memory
> 
> 
> Well, I'm reticent to suggest something going on under the hood without
> taking a good look at it, however, if there is something wrong in the broker
> then I presume it has to do with a "sweeper"-like thread that periodically
> cleans up queues and topics that are left pending and consumerless (similar
> to how sweeper threads would clean up expired messages etc.
> 
> Then again, we are the only two that I know of describing this behaviour.
> This suggests that either there is something bizarrely odd about our
> configurations or we are, indeed, correct :)
> 
> 
> 
> Frugia, Kirby A wrote:
>> 
>> Yes, this is exactly what we are seeing.  Any ideas?
>> 
>> -----Original Message-----
>> From: Will van der Leij [mailto:will@stonethree.com]
>> Sent: Thursday, November 27, 2008 11:42 PM
>> To: users@activemq.apache.org
>> Subject: Re: Broker not releasing memory
>> 
>> 
>> We see similar behaviour in a slightly different setup so I'd like to
>> elaborate a bit in the hope of finding out if this is either intended
>> behaviour or a memory leak.
>> 
>> Essentially, with a single or multiple producers and a single or multiple
>> consumers on a single topic:
>>  - We send messages faster than the consumer can read asynchronously
>>  - When the send queue is larger than the consumer's prefetch buffer then
>>    we correctly see messages filling up the broker memory
>> (memoryPercentUSage in JMX)
>>  - Killing the consumer (cleanly through a close or not-cleanly with a
>> terminate signal)
>>    results in the memoryPercentUsage not being released
>>    - even though there are no more consumers on the Topic
>>    - this memory is never released until the broker is restarted
>> 
>> I've tried it with persistence on and off, all the retroactive recovery
>> options on and off, flow control on and off, caching on and off,
>> asynchronous dispatching on and off, noLocal flag on and off etc.
>> 
>> This is a very typical case of a consumer dying with inflight messages on
>> the broker. Is the behaviour of the messages not being cleaned up or
>> recovered intended (i.e. they are there for a reason) or is there soething
>> wrnog with the setup.
>> 
>> Many thanks
>> Will van der Leij
>> 
>> 
>> 
>> Frugia, Kirby A wrote:
>>> 
>>> Hi All,
>>> 
>>> Sorry I dual-posted in the dev list.  I think I sent to the wrong one...
>>> 
>>> We are seeing an issue with our broker not releasing memory on topics.
>>> 
>>> Setup:
>>> * Active MQ 5.1 (out of the box)
>>> * Persistent messages sent by publishers
>>> * Non-durable topics
>>> 
>>> We were seeing an issue in production with slow consumers causing the
>>> broker to run out of memory.
>>> 
>>> We wrote a test app, which has one publisher and one subscriber to the
>>> same topic.  The publisher sends messages on this topic frequently.  The
>>> subscriber, upon receiving a message, goes to sleep for 10 minutes.
>>> 
>>> Very quickly, the broker runs out of memory.  When this happens, our
>>> publishers can no longer send messages; the send is blocked. This is
>>> expected.  However, if we kill our application (which cleanly disconnects
>>> from the broker), the broker's memory usage (MemoryPercentUsage) does not
>>> go back down.  Also, any new apps that startup will have their publishers
>>> blocked.
>>> 
>>> Shouldn't the broker release the memory associated with these messages?
>>> 
>>> Thanks,
>>> Kirby
>>> 
>>> 
>>> 
>> 
>> --
>> View this message in context:
>> http://www.nabble.com/Broker-not-releasing-memory-tp20662078p20730116.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> 
>> 
>> 
> 
> --
> View this message in context:
> http://www.nabble.com/Broker-not-releasing-memory-tp20662078p20808585.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> 


Mime
View raw message