activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philippe Mouawad (JIRA)" <>
Subject [jira] Commented: (AMQ-2533) OutOfMemory provoked by Advisory messages
Date Fri, 08 Jan 2010 11:00:15 GMT


Philippe Mouawad commented on AMQ-2533:

I attached 3 files that enable repdrocing the OOM due to Advisory messages.
I think that issue is due to the fact that advisory messages are generating FULL Advisory
either due to error in MemoryUsage computation based on parent (as I understant in code).
Thus when many Full Advisory messages are sent, Advisory end up generating advisory for advisory
cascading this and ending with OOM 

> OutOfMemory provoked by Advisory messages
> -----------------------------------------
>                 Key: AMQ-2533
>                 URL:
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.2.0
>            Reporter: Philippe Mouawad
>            Priority: Critical
>         Attachments: activemq-OOM-ADVISORY.xml, activemq.xml, Image 11.png, ThreadDump.txt,,
> Hello,
> We are in an environment where we have 1 Broker and 6 clients listener to one Topic.
> We enabled advisory messages through:
> <policyEntry topic=">" memoryLimit="5mb" producerFlowControl="true"  advisoryWhenFull="true"
>                       sendAdvisoryIfNoConsumers="true">
> And sendFailIfNoSpace:
> <systemUsage>
>             <systemUsage sendFailIfNoSpace="true">
>                 <memoryUsage>
>                     <memoryUsage limit="50 mb"/>
>                 </memoryUsage>
>                 <storeUsage>
>                     <storeUsage limit="1 gb" name="foo" store="#store"/>
>                 </storeUsage>
>                 <tempUsage>
>                     <tempUsage limit="100 mb"/>
>                 </tempUsage>
>             </systemUsage>
>         </systemUsage>
> We also have a monitoring process that listens to advisory messages related to the concerned
> After some run, we end up with OutOfMemory.
> OutOfMemory analysis shows:
> - 3372 instances of org.apache.activemq.command.ActiveMQTopic occupying 23,7%
> Names of Topic contain ActiveMQ.Advisory.FULL.Topic:
> - ActiveMQ.Advisory.FULL.Topic.ActiveMQ.Advisory.FULL.Topic.ActiveMQ.Advisory.FULL.....ActiveMQ.Advisory.Producer.Topic.OFBTopic
> 1069 instances of org.apache.activemq.thread.DedicatedTaskRunner$1 holding 12% through
the names of the thread that are soooooo long.
> Debuging ActiveMQ, I noticed that AMQ didn't check when sending advisory messages that
advisory concerns an advisory.
> Example, suppose you set a listener for advisory message ActiveMQ.Advisory.FULL.Topic.OFBTopic
and this listener becomes slow for a reason, then AMQ will send an advisory on this advisory
provoking more and more traffic 
> on the broker only for this advisory.
> I attached a Stack Trace showing that an advisory will be sent for the advisory.
> I will be checking to see if this issue also exists on 5.3.0.
> Thanks in advance.
> Philippe.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message