activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-4319) ActiveMQ connection info cached and never cleaned up, causes ActiveMQ to crash.
Date Thu, 14 Feb 2013 18:33:13 GMT

    [ https://issues.apache.org/jira/browse/AMQ-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13578556#comment-13578556
] 

Gary Tully commented on AMQ-4319:
---------------------------------

the problem is the org.apache.activemq.security.AuthorizationBroker, it adds destination to
the security context on addProducer/addConsumer but when these destinations are gc'ed they
are not removed from the connection context.
With a long lived connection, these can accumulate.

securityContext.getAuthorizedReadDests().put(info.getDestination(), info.getDestination());

the fix is probably in org.apache.activemq.security.AuthorizationBroker#removeDestination,
where we need to remove the destination from the auth map of all connections.

org.apache.activemq.security.SimpleAuthenticationPluginTest would be a simple place to add
a test case, once the gcInactive is added to org/apache/activemq/security/simple-auth-broker.xml

creating producers/consumers to USERS.1-1000 and having them gced and checking jvm stack usage
should expose the problem in a unit test.
                
> ActiveMQ connection info cached and never cleaned up, causes ActiveMQ to crash.
> -------------------------------------------------------------------------------
>
>                 Key: AMQ-4319
>                 URL: https://issues.apache.org/jira/browse/AMQ-4319
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.6.0
>         Environment: RHEL 6
>            Reporter: Matt Woodson
>
> We are using ActiveMQ + mcollective.  With mcollective, we are using the "activemq connector
([http://docs.puppetlabs.com/mcollective/reference/plugins/connector_activemq.html])"
> We are running ActiveMQ in the EC2 Cloud, so we have enabled security for the connections
being made.
> We have 2 ActiveMQ servers that are configures as network of brokers, about around 200
nodes that we communicate with.
> The problem we are seeing is that after about 1 week in operation, ActiveMQ will crash
because it runs out of memory.  We have been monitoring the heap, and you will see it rise
during the week, and then the wrapper will eventually restart activemq
> {code:title=wrapper.log}
> ERROR  | wrapper  | 2013/02/01 09:38:33 | JVM appears hung: Timed out waiting for signal
from JVM.
> ERROR  | wrapper  | 2013/02/01 09:38:33 | JVM did not exit on request, terminated
> INFO   | wrapper  | 2013/02/01 09:38:33 | JVM exited on its own while waiting to kill
the application.
> STATUS | wrapper  | 2013/02/01 09:38:33 | JVM exited in response to signal SIGKILL (9).
> STATUS | wrapper  | 2013/02/01 09:38:38 | Launching a JVM...}
> {code}
> In the activemq connector page, it says that there needs to be cleanup of queues.  This
is put into place.
> I did a heap dump, and with external help I was told that "the authentication plugin
is retaining destinations in a cached connection"
> These connections need to be deleted and cleaned up as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message