activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "matteo rulli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-4889) ProxyConnector memory usage skyrockets when several ssl handshakes fails
Date Fri, 29 Nov 2013 11:32:37 GMT

    [ https://issues.apache.org/jira/browse/AMQ-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13835322#comment-13835322
] 

matteo rulli commented on AMQ-4889:
-----------------------------------

Actually the problem in ProxyConnector hid something more than a simple problem ProxyConnector.java
file. 

Apparently some clean-up invocations were missing in SSLNIOTransport as well. This resulted
in many leaked file descriptors in linux (RHEL 6.3 64 bit). 

Before the attached patches the lsof command gave tons of
{noformat}
java    6530 platone    8u  sock                0,6      0t0 2489682586 can't identify protocol
java    6530 platone    9u  sock                0,6      0t0 2489745273 can't identify protocol
java    6530 platone   15u  sock                0,6      0t0 2489727545 can't identify protocol
java    6530 platone   66u  sock                0,6      0t0 2489683982 can't identify protocol
java    6530 platone   69u  sock                0,6      0t0 2489684335 can't identify protocol
java    6530 platone   72u  sock                0,6      0t0 2489684339 can't identify protocol
java    6530 platone   74u  sock                0,6      0t0 2489684688 can't identify protocol
java    6530 platone   76u  sock                0,6      0t0 2489685055 can't identify protocol
java    6530 platone   77u  sock                0,6      0t0 2489716539 can't identify protocol
java    6530 platone   97u  sock                0,6      0t0 2489683431 can't identify protocol
java    6530 platone   98u  sock                0,6      0t0 2489683450 can't identify protocol
java    6530 platone   99u  sock                0,6      0t0 2489684695 can't identify protocol
java    6530 platone  100u  sock                0,6      0t0 2489683990 can't identify protocol
java    6530 platone  101u  sock                0,6      0t0 2489702245 can't identify protocol
java    6530 platone  102u  sock                0,6      0t0 2489685058 can't identify protocol
...
{noformat}
(see attached lsof.txt) with the following socket stats:
{noformat}
sockets: used 3067
TCP: inuse 21 orphan 0 tw 0 alloc 2808 mem 2648
UDP: inuse 6 mem 5
UDPLITE: inuse 0
RAW: inuse 1
FRAG: inuse 0 memory 0
{noformat}
and the following netstat outcome:
{noformat}
...
tcp        0      0 ::ffff:192.168.24.82:44201  ::ffff:192.168.16.166:61616 ESTABLISHED 6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.66:64601  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.67:9033   CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.66:56924  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.66:33021  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.67:56879  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.67:51487  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.66:35295  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.67:49529  CLOSE_WAIT  6530/java
          off (0.00/0/0)
tcp        0      0 ::ffff:192.168.24.82:61619  ::ffff:192.168.24.67:51309  CLOSE_WAIT  6530/java
          off (0.00/0/0)
... many other CLOSE_WAIT
{noformat}
We applied the attached patches and everything run better: the lsof stopped reporting the
_can't identify protocol_ records and the number of connections in CLOSE_WAIT dropped to zero.

So what? Does it make sense to you?

> ProxyConnector memory usage skyrockets when several ssl handshakes fails
> ------------------------------------------------------------------------
>
>                 Key: AMQ-4889
>                 URL: https://issues.apache.org/jira/browse/AMQ-4889
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.8.0, 5.9.0
>         Environment: Seen in Windows 7 64bit, Windows Server 2008 R2 and Linux RHEL 6.3
64 bit
>            Reporter: matteo rulli
>         Attachments: ProxyConnIssue.rar
>
>
> See [nabble|http://activemq.2283324.n4.nabble.com/Proxy-Connector-memory-consumption-td4674255.html]
for further details.
> To reproduce the issue:
> # Start embedded proxy broker and the AMQ broker that are embedded in *AMQTestBroker*
project (see attachments);
> # Start the *AMQTestConsumer* project; This program repeatedly tries opening a connection
to the ProxyConnector with wrong certificates.
> # Open jconsole to monitor AMQTestBroker memory usage: you should experience an OOM error
within one hour with the suggested settings (Xmx = 2048m).
> Launch configurations and test keystores are attached to this issue along with the java
projects.
> This behavior seems to affect _ProxyConnector_ only, running the test against a standard
nio-based _TransportConnector_ does not seem to produce anomalous memory consumptions.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message