activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Wuwer (JIRA)" <j...@apache.org>
Subject [jira] Created: (AMQ-2596) Memory leak in network of brokers - the number of TopicSubscription/ConsumerInfo objects increases
Date Sun, 07 Feb 2010 14:06:32 GMT
Memory leak in network of brokers - the number of TopicSubscription/ConsumerInfo objects increases
--------------------------------------------------------------------------------------------------

                 Key: AMQ-2596
                 URL: https://issues.apache.org/activemq/browse/AMQ-2596
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker
    Affects Versions: 5.3.0, 4.1.1
         Environment: OS: SLES10 SP2 or Ubuntu 9.10 (my desktop)
JVM: IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20081129 (JIT enabled)
            Reporter: Mark Wuwer
            Priority: Critical


I'd like to report a reproducible memory leak in network of brokers.

This is the scenario you can reproduce it (tried out with 4.1.1, 5.3.0 and 5.3.1):
1. start two brokers using attached configuration.
    Lets call one of it BE (for backend) another FE (for frontend) - accordingly: be_activemq.xml
and fe_activemq.xml are attached.
    The BE broker contains following configuration: 
        <transportConnectors>
            <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
        </transportConnectors>
        <networkConnectors>
           <networkConnector name="FE_connector" dynamicOnly="true" uri="static://(tcp://localhost:61617)"/>
        </networkConnectors>

    The FE broker contains just:
        <transportConnectors>
            <transportConnector name="openwire" uri="tcp://0.0.0.0:61617"/>
        </transportConnectors>

2. now start a client that opens and close permanently topic subscriptions (see attached tests)
    to the FE broker

3. after a while you will see (e.g. verbose gc) that the heap usage of the BE broker will
increase,
    but on FE not

If you analyze the heap dump of BE you will see that the memory leak consist of a large number
of ConsumerInfo
objects. Easy way to find the objects when you use MAT is to use the OQL query e.g.:
    SELECT c.destination.physicalName.toString() FROM ".*ConsumerInfo" c WHERE (c.destination.physicalName.toString()
LIKE "TEST_TOPIC")

I noticed that basically for each subscription on FE two ConsumerInfo objects are created
on BE, which will never be closed.
There are however some special cases that only 4, 8 .. 12 objects per connection are created.
This happens when the client just opens sessions/subscriptions without closing  (see appropriate
junit test with comments in it).

We have it currently a P1 ticket at our customer site (running 3*FE + BE using 4.1.1) because
of OoM that occurs periodically (in fact we now restart  every n-days the system). 

Another strange issue that happens at our customer site is something that I cannot reproduce
at all.
The ConsumerInfo objects on BE contains there non empty selector strings, which makes the
memory leak more critical.
In my test environment however selector strings are only set in ConsumerInfo on FE, but in
BE broker the selectorin ConsumerInfo is always null!
I compared the customer configuration with my and I could not find any difference.

Any idea what could influence this behavior?

Thanks in advance for any feedback and help on this issue.
Best Regards,
Mark

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message