curator-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evan Pollan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CURATOR-354) GzipCompressionProvider leaks native memory
Date Thu, 06 Oct 2016 20:45:20 GMT

    [ https://issues.apache.org/jira/browse/CURATOR-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553130#comment-15553130
] 

Evan Pollan commented on CURATOR-354:
-------------------------------------

I've attached a screenshot of DataDog showing RSS for two containers running a JVM with curator
being used to read/write some compressed znodes.  Prior to the fix, the JVMs (which have 2G
max heaps) would get OOM killed when the JVM allocated all of the 3.5 GB of RAM  in the docker
container.  You can see after the fix, that RSS stabilizes at somewhere around 2.25 GB and
stays flat.

> GzipCompressionProvider leaks native memory
> -------------------------------------------
>
>                 Key: CURATOR-354
>                 URL: https://issues.apache.org/jira/browse/CURATOR-354
>             Project: Apache Curator
>          Issue Type: Bug
>    Affects Versions: 2.11.0
>            Reporter: Evan Pollan
>            Assignee: Jordan Zimmerman
>         Attachments: RSS before and after fix.png
>
>
> Both compress and decompress instantiate GZIP streams, use them, and don't close them.
 The java util deflate/inflate streams use JNI to leverage zlib, including the allocation
of a native, off-heap buffer.  
> These JNI buffer handles are released when the containing stream is closed.  They're
also released by the finalize() method on java.util.zip.Inflater/Deflater, _but_ it looks
as if the native memory can pile up quickly enough in certain use cases where the JVM is OOM
killed before the finalizer does its thing.
> Fix is to explicitly close these streams after they're used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message