activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Dillon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-4986) tightEncoding=false + failover triggers Broker memory exhaust
Date Mon, 24 Nov 2014 16:55:12 GMT

    [ https://issues.apache.org/jira/browse/AMQ-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223128#comment-14223128
] 

Michael Dillon commented on AMQ-4986:
-------------------------------------

We are having similar issues as described above.  The clients are running the 5.10 library
and the broker is also 5.10.  We see the client subscriber reconnecting every 30 seconds.
 The difference is that we have the default setting for tightEncodingEnabled=true.  As a test
we set this to false on our client and the reconnecting issue went away.  Our concern now
is that we will experience the OutOfMemory issue as described above.  It would be great if
we could get some guidance on this issue.  

> tightEncoding=false + failover triggers Broker memory exhaust
> -------------------------------------------------------------
>
>                 Key: AMQ-4986
>                 URL: https://issues.apache.org/jira/browse/AMQ-4986
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.6.0, 5.7.0, 5.8.0, 5.9.0
>         Environment: java 1.7
> Client library 5.5.1
>            Reporter: Felix Ehm
>             Fix For: NEEDS_REVIEW
>
>
> We experience this problem in combination with 5.5.1 client and the wireformat.tightEncodingEnabled=false+
failover:
> Scenario:
> 1. start standard broker
> 2. start Client (with e.g. a MessageListener) with failover protocol: e.g. failover:(tcp://123.123.123.123:61616?wireformat.tightEncodingEnabled=false)
> 3. wait around 30sec (default for inactivity check)
> Result:
> The client closes the connection and re-tries to the broker which in turn throws the
following exception:
> {code}
> 2014-01-21 20:12:49,568 [ActiveMQ Transport: tcp:///123.123.123.123:60156@61616] DEBUG
Transport  Transport Connection to: tcp://124.124.124.124:60156 failed: java.io.IOException:
Unexpected error occured: java.lang.OutOfMemoryError: Java heap space
> java.io.IOException: Unexpected error occured: java.lang.OutOfMemoryError: Java heap
space
> 	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:203)
> 	at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.activemq.openwire.v8.BaseDataStreamMarshaller.looseUnmarshalByteSequence(BaseDataStreamMarshaller.java:638)
> 	at org.apache.activemq.openwire.v8.WireFormatInfoMarshaller.looseUnmarshal(WireFormatInfoMarshaller.java:132)
> 	at org.apache.activemq.openwire.OpenWireFormat.doUnmarshal(OpenWireFormat.java:373)
> 	at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:285)
> 	at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
> 	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
> 	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
> 	... 1 more
> {code}
> The problem here is that the BaseDataStreamMarshaller reads an int from the buffer and
re-uses it immediately to allocate a byte array:
> {code}
> protected byte[] looseUnmarshalByteArray(DataInput dataIn) throws IOException {
>         byte rc[] = null;
>         if (dataIn.readBoolean()) {
>             int size = dataIn.readInt();
>             rc = new byte[size];   // PROBLEM! What happens if size has been read and
interpreted wrongly ? 
>             dataIn.readFully(rc);
>         }
>         return rc;
>     }
> {code}
> In our case the dadaIn.readInt() read an int number of 785.477.224 which triggers the
broker to allocate blindly this amount of mem.
> We do not know yet what triggers the wrong byte sequence from the client, but on the
brokers side, there should be a protection against this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message