activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Gies <ag...@progress.com>
Subject Re: Use of MirroredQueues results in OutOfMemoryError
Date Fri, 27 Mar 2009 16:27:03 GMT
Hello,

I have just rerun the testcase against the latest trunk and the  
problem does not occur anymore. I have let the test run
for about a million messages.

Ig you have time, perhaps you can double check this ?

Best regards
Andreas


On Mar 26, 2009, at 5:41 PM, Andreas Gies wrote:

> Hi there,
>
> I apologize for not heaving read all the information from the  
> beginning - I shouldn't answer mails while traveling ;)
> I could reproduce the problem using your test case in both AMQ 5.2  
> and the current trunk.
>
> I have made the same observations regarding the objects that are  
> growing in number. As a side issue, the reported
> number via jconsole regarding the Queuesize is misleading. There are  
> actually no messages available for delivery
> even if the queue size shows a number > 0. I believe there is an  
> open issue regarding that misleading metric:
> http://issues.apache.org/activemq/browse/AMQ-1600
>
> I think that explains why you see a queue size > 0.
>
> This having said, a very brief look at the code, the interceptor  
> dealing with Mirrored queues simply recreates
> the send ProducerExchange and calls the delivery for that. Why the  
> memory leak occurs here, but not with
> a "normal" publisher to a topic with no subscribers, is not yet  
> clear to me. Perhaps someone else can jump in here.
>
> I think for now it would be best to create a JIRA and attach your  
> test case if possible, so that we can track that properly.
>
>
> Best regards
> Andreas
>
>
>
> On Mar 25, 2009, at 3:00 PM, jvh wrote:
>
>>
>>
>> We're using activemq-all-5.2.0.jar ... more information is listed  
>> at the
>> bottom of the original post.
>>
>>
>> Andreas Gies-3 wrote:
>>>
>>> Hi there,
>>>
>>> which version of Active MQ are you using ?
>>>
>>>
>>> Regards
>>> Andreas
>>> On Mar 24, 2009, at 9:20 PM, jvh wrote:
>>>
>>>>
>>>> We are experiencing an OutOfMemoryError when using MirroredQueues
>>>> and I was
>>>> wondering if someone could comment on whether our use (or
>>>> interpretation) of
>>>> these Mirrored Queues is correct, or if there is a bug in the
>>>> BrokerService
>>>> code.
>>>>
>>>> In our case, we have an existing application “A” that uses
>>>> messaging, but we
>>>> now have a new application “B” that will only sometimes monitor the
>>>> messaging on the original application “A”.  We don’t want the first
>>>> application “A” to have to know about, or to do anything  
>>>> different if
>>>> application “B” is there listening to the messages or not.
>>>>
>>>> This seemed like a decent use of MirroredQueues (since they
>>>> evidently create
>>>> a “topic” that can be used as a wire-tap), but evidently by setting
>>>> BrokerService.setUseMirroredQueues(true) on the broker for Project
>>>> “A”,
>>>> we’ve introduced a rather large memory leak.
>>>>
>>>> The objects that appear to grow in number are:
>>>> org.apache.activemq.store.memory.MemoryTransactionStore$2
>>>> org.apache.activemq.store.memory.MemoryTopicMessageStore
>>>> org.apache.activemq.command.ActiveMQTopic
>>>>
>>>> This seems to suggest the mirrored messages are being retained, but
>>>> the
>>>> Broker has setPersistent(false) specified and I don’t see any way  
>>>> to
>>>> set
>>>> Time to Live, or any other way for the Broker to monitor and  
>>>> remove if
>>>> necessary on the Mirrored Queue only, etc..  If I look at the  
>>>> queues
>>>> via
>>>> jConsole MBeans, I can see the
>>>> Topic.VirtualTopic.Mirror.DacIncomingQueue
>>>> with attributes showing that the messages are only being enqueue  
>>>> and
>>>> not
>>>> dequeued.  I really want the Broker to drop these messages if
>>>> Application
>>>> “B” is not there to dequeue them.
>>>>
>>>> Is it recommended that we use MirroredQueues to do this?  If so,
>>>> what’s the
>>>> best way to set things up so I don’t have a memory problem if the
>>>> other
>>>> application is not listening to the mirroredQueue? … and if not, we
>>>> would
>>>> appreciate any other suggestions.
>>>>
>>>> ---- For attached Example Code----
>>>> The code attached is a simplified version of our Project “A” only  
>>>> and
>>>> represents the case where Project “B” is not online (and not
>>>> included in
>>>> this example at all in fact) … it does show our use of Spring
>>>> Templates to
>>>> set up the Broker (and the producer and consumer clients).  I’ve
>>>> tried to
>>>> remove quite a bit of the extraneous stuff to try to get it down to
>>>> the
>>>> basic problem, but I apologize that it still has a bit of fluff.
>>>> I’ve also
>>>> arranged things such that the broker is in a separate JVM to more
>>>> easily see
>>>> the memory growth of this component.  The BrokerService is  
>>>> specified
>>>> in the
>>>> jms.broker.BrokerBean class.
>>>>
>>>> For the attached code and the 8MB specified for the JVM, we get
>>>> about 3542
>>>> messages before the broker dies with an OutOfMemoryError
>>>>
>>>> I’ve also noticed that there is a
>>>> BrokerService.setUseTempMirroredQueue()
>>>> method and wonder if that is more appropriate for my use since it
>>>> implies
>>>> that the Mirrored Queue will be temporary, but the JavaDoc for this
>>>> message
>>>> is empty and I’ve seen nothing on the forums as to its
>>>> characteristics and
>>>> proper use.  When I do use the Temporary Mirrored Queues, I also
>>>> don’t see
>>>> the same Virtual Topic that should be created that I see with the
>>>> regular
>>>> MirroredQueue topic (i.e. I don’t see VirtualTopic.Mirror.Foo.Bar
>>>> being
>>>> created for a queue of Foo.Bar), so this doesn’t seem to fit our
>>>> needs.
>>>>
>>>> If you want to run the code, run jms-broker first, then run jms- 
>>>> client
>>>> (client contains both a message producer and consumer).  There is a
>>>> README.txt file in the top level directory of each application for
>>>> build and
>>>> execution instructions.
>>>>
>>>> Using ActiveMQ 5.2.0 and Spring 2.5.4 on a Windows XP SP2 platform
>>>> (but
>>>> we’ve seen this on unix platforms as well) and I’ve been using
>>>> jconsole, and
>>>> jvisualvm to monitor memory growth and queues
>>>>
>>>> http://www.nabble.com/file/p22688889/jmsMirroredQueueExample.zip
>>>> jmsMirroredQueueExample.zip
>>>> -- 
>>>> View this message in context:
>>>> http://www.nabble.com/Use-of-MirroredQueues-results-in-OutOfMemoryError-tp22688889p22688889.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>>
>>> ---
>>> Mit freundlichen Grüssen - Kind Regards
>>> Andreas Gies
>>> Principal Consultant
>>> Open Source Center of Competence
>>>
>>> Progress Software GmbH
>>> Agrippinawerft 26
>>> 50678 Köln
>>>
>>> E-Mail      	agies@progress.com
>>> Direct Line 	+49 (0)9953 980349
>>> Mobile      	+49 (0)170 5759611
>>> Skype        	+44 (0)20 3239 2922
>>> Skype       	+353 (0)1 443 4971
>>> Skype       	+1 (0)781 262 0168
>>>
>>> http://www.progress.com
>>> http://fusesource.com
>>> http://open-source-adventures.blogspot.com
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> Progress Software GmbH
>>> Sitz der Gesellschaft: Agrippinawerft 26, 50678 Koeln;
>>> Niederlassung: Fuerstenrieder Str. 279, 81377 Muenchen
>>> Amtsgericht Koeln, HRB 15620;
>>> Geschaeftsfuehrung: David Ireland
>>> -------------------------------------------------------
>>>
>>>
>>
>> -- 
>> View this message in context: http://www.nabble.com/Use-of-MirroredQueues-results-in-OutOfMemoryError-tp22688889p22702274.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
> ---
> Mit freundlichen Grüssen - Kind Regards
> Andreas Gies
> Principal Consultant
> Open Source Center of Competence
>
> Progress Software GmbH
> Agrippinawerft 26
> 50678 Köln
>
> E-Mail      	agies@progress.com
> Direct Line 	+49 (0)9953 980349
> Mobile      	+49 (0)170 5759611
> Skype        	+44 (0)20 3239 2922
> Skype       	+353 (0)1 443 4971
> Skype       	+1 (0)781 262 0168
>
> http://www.progress.com
> http://fusesource.com
> http://open-source-adventures.blogspot.com
>
>
>
> -------------------------------------------------------
> Progress Software GmbH
> Sitz der Gesellschaft: Agrippinawerft 26, 50678 Koeln;
> Niederlassung: Fuerstenrieder Str. 279, 81377 Muenchen
> Amtsgericht Koeln, HRB 15620;
> Geschaeftsfuehrung: David Ireland
> -------------------------------------------------------

---
Mit freundlichen Grüssen - Kind Regards
Andreas Gies
Principal Consultant
Open Source Center of Competence

Progress Software GmbH
Agrippinawerft 26
50678 Köln

E-Mail      	agies@progress.com
Direct Line 	+49 (0)9953 980349
Mobile      	+49 (0)170 5759611
Skype        	+44 (0)20 3239 2922
Skype       	+353 (0)1 443 4971
Skype       	+1 (0)781 262 0168

http://www.progress.com
http://fusesource.com
http://open-source-adventures.blogspot.com



-------------------------------------------------------
Progress Software GmbH
Sitz der Gesellschaft: Agrippinawerft 26, 50678 Koeln;
Niederlassung: Fuerstenrieder Str. 279, 81377 Muenchen
Amtsgericht Koeln, HRB 15620; 
Geschaeftsfuehrung: David Ireland
-------------------------------------------------------

Mime
View raw message