curator-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (CURATOR-487) GzipCompressionProvider produces a lot of finalizable objects
Date Mon, 10 Dec 2018 15:24:00 GMT


ASF GitHub Bot commented on CURATOR-487:

Github user leventov commented on the issue:
    There is no peer evidence here, because we are on the optimization forefront. See
    I've seen a Druid heap with more than 10k finalizable Deflater objects, about 8k of which
were already dead, awaiting in the finalization queue. They come from `GzipCompressionProvider`.
    Historically Druid uses Zookeeper somewhat wrong (not for what Zookeeper was designed):
it announces data segment placement using Zookeeper, that leads to creation of a lot of new
nodes in Zookeeper every second. It means that by accident, Druid is a good stress test for
Zookeeper (and consequently for Curator), and we run probably the largest Druid cluster.

> GzipCompressionProvider produces a lot of finalizable objects
> -------------------------------------------------------------
>                 Key: CURATOR-487
>                 URL:
>             Project: Apache Curator
>          Issue Type: Improvement
>            Reporter: Roman Leventov
>            Assignee: Cameron McKenzie
>            Priority: Major
>             Fix For: 4.1.0
> GzipCompressionProvider.compress() and decompress() methods are called a lot inside and
outside of the framework and each such call produces a or Inflater
object that are finalizable, that is bad for GC.
> Compressing or decompressing a finite byte[] object (in contrast to compressing/decompressing
an InputStream or an OutputStream of unknown length) is actually a happy case because even
PhantomReference object could be avoided, the native resources could be created and freed
in a single try-finally block.
> Curator must avoid that. [] is a potential replacement,
for example.
> A similar issue could be also fixed in Apache Commons Compress: and
Curator is made to depend on Commons Compress.

This message was sent by Atlassian JIRA

View raw message