hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5879) GzipCodec should read compression level etc from configuration
Date Sat, 18 Jul 2009 01:50:14 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Douglas updated HADOOP-5879:

    Assignee: He Yongqiang
      Status: Open  (was: Patch Available)

The latest patch looks good. A few notes:
* BuiltInZlibDeflater can use the setLevel and setStrategy methods on java.util.zip.Deflater
to implement reset
* This should definitely have a unit test, particularly involving CodecPool
* The javadoc for reinit should describe the contract for that method; the implementations
should also describe what happens for that particular codec.
* In ZlibCompression::reinit, shouldn't end() be called before reconfiguring the codec?
* The direct buffer size should not be reconfigured. Codecs are pooled because recreating
direct buffers is very expensive; allocating new direct buffers on a cache hit defeats the
purpose of pooling them. The buffer size doesn't affect correctness of zlib compression.
* This should add methods like ZlibCompression::setCompressionLevel(Configuration, <value>)
to set the {{zlib.compress.*}} properties

> GzipCodec should read compression level etc from configuration
> --------------------------------------------------------------
>                 Key: HADOOP-5879
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5879
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>            Reporter: Zheng Shao
>            Assignee: He Yongqiang
>         Attachments: hadoop-5879-5-21.patch, hadoop-5879-7-13-2.patch, hadoop-5879-7-13-3.patch
> GzipCodec currently uses the default compression level. We should allow overriding the
default value from Configuration.
> {code}
>   static final class GzipZlibCompressor extends ZlibCompressor {
>     public GzipZlibCompressor() {
>       super(ZlibCompressor.CompressionLevel.DEFAULT_COMPRESSION,
>           ZlibCompressor.CompressionStrategy.DEFAULT_STRATEGY,
>           ZlibCompressor.CompressionHeader.GZIP_FORMAT, 64*1024);
>     }
>   }
> {code}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message