curator-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evan Pollan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CURATOR-354) GzipCompressionProvider leaks native memory
Date Thu, 06 Oct 2016 20:45:20 GMT

     [ https://issues.apache.org/jira/browse/CURATOR-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Evan Pollan updated CURATOR-354:
--------------------------------
    Description: 
Both compress and decompress instantiate GZIP streams, use them, and don't close them.  The
java util deflate/inflate streams use JNI to leverage zlib, including the allocation of a
native, off-heap buffer.  

These JNI buffer handles are released when the containing stream is closed.  They're also
released by the finalize() method on java.util.zip.Inflater/Deflater, _but_ it looks as if
the native memory can pile up quickly enough in certain use cases where the JVM is OOM killed
before the finalizer does its thing.

Fix is to explicitly close these streams after they're used.

This was discovered in version 2.11.0, but it looks like the problem affects all versions
for which {{GzipCompressionProvider}} existed.



  was:
Both compress and decompress instantiate GZIP streams, use them, and don't close them.  The
java util deflate/inflate streams use JNI to leverage zlib, including the allocation of a
native, off-heap buffer.  

These JNI buffer handles are released when the containing stream is closed.  They're also
released by the finalize() method on java.util.zip.Inflater/Deflater, _but_ it looks as if
the native memory can pile up quickly enough in certain use cases where the JVM is OOM killed
before the finalizer does its thing.

Fix is to explicitly close these streams after they're used.




> GzipCompressionProvider leaks native memory
> -------------------------------------------
>
>                 Key: CURATOR-354
>                 URL: https://issues.apache.org/jira/browse/CURATOR-354
>             Project: Apache Curator
>          Issue Type: Bug
>    Affects Versions: 2.11.0
>            Reporter: Evan Pollan
>            Assignee: Jordan Zimmerman
>         Attachments: RSS before and after fix.png
>
>
> Both compress and decompress instantiate GZIP streams, use them, and don't close them.
 The java util deflate/inflate streams use JNI to leverage zlib, including the allocation
of a native, off-heap buffer.  
> These JNI buffer handles are released when the containing stream is closed.  They're
also released by the finalize() method on java.util.zip.Inflater/Deflater, _but_ it looks
as if the native memory can pile up quickly enough in certain use cases where the JVM is OOM
killed before the finalizer does its thing.
> Fix is to explicitly close these streams after they're used.
> This was discovered in version 2.11.0, but it looks like the problem affects all versions
for which {{GzipCompressionProvider}} existed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message