cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hannes Schmidt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2654) Work around native heap leak in sun.nio.ch.Util affecting IncomingTcpConnection
Date Sun, 15 May 2011 21:48:47 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033824#comment-13033824
] 

Hannes Schmidt commented on CASSANDRA-2654:
-------------------------------------------


I think v2 has issues: it has an out-of-bounds condition it invokes readFully redundantly.


Let's say size is 1, then v2 does this:

contentBytes = new byte[1];
input.readFully(contentBytes, 0, CHUNK_SIZE); // oob
input.readFully(contentBytes, 0, 1); // redundant 

If size is CHUNK_SIZE it does this

contentBytes = new byte[CHUNK_SIZE];
input.readFully(contentBytes, 0, CHUNK_SIZE);
input.readFully(contentBytes, CHUNK_SIZE, 0); // redundant, 3rd argument is 0

If size is CHUNK_SIZE+1 it does this

contentBytes = new byte[CHUNK_SIZE+1];
input.readFully(contentBytes, 0, CHUNK_SIZE);
input.readFully(contentBytes, CHUNK_SIZE, CHUNK_SIZE); // oob
input.readFully(contentBytes, CHUNK_SIZE, 1); // redundant

DataInputStream.readFully(a,o,l) actually prevents the OOB but it's not good style to rely
on that. The redundant invocations do matter when size is near CHUNK_SIZE.

> Work around native heap leak in sun.nio.ch.Util affecting IncomingTcpConnection
> -------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-2654
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2654
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.6.13, 0.7.5, 0.8.0 beta 2
>         Environment: OpenJDK Runtime Environment (IcedTea6 1.9.7) (6b20-1.9.7-0ubuntu1~10.04.1)
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
> Also observed on Sun/Oracle JDK. Probably platform- and os-independent.
>            Reporter: Hannes Schmidt
>             Fix For: 0.6.12
>
>         Attachments: 2654-v2.txt, chunking.diff
>
>
> NIO's leaky, per-thread caching of direct buffers in combination with IncomingTcpConnection's
eager buffering of messages leads to leakage of large amounts of native heap. Details in [1].
More on the root cause in [2]. Even though it doesn't fix the leak, attached patch has been
found to alleviate the problem by keeping the size of each direct buffer modest.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message