hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5879) GzipCodec should read compression level etc from configuration
Date Wed, 27 May 2009 06:49:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713446#action_12713446
] 

He Yongqiang commented on HADOOP-5879:
--------------------------------------

test failed test is not related this patch.
However, what Chris commented is right. The current patch can not guarantee the Gzip Compressor
got from CodecPool is of the settings what users expect.  This is kind of global settings,
but if we change settings but the CodecPool does not clean its buffered codecs which of old
settings. So it may make things work in a wrong way. 
So one possible way is to let CodecPool do special for Gzip codec, and does either 
1) keeps a map for holding gzip codec of different settings.
or
2) treats the setting as a global setting, and when the setting is changed, clean all gzip
codecs cached in CodecPool.

Does the changes for CodecPool sound reasonable/acceptable?

> GzipCodec should read compression level etc from configuration
> --------------------------------------------------------------
>
>                 Key: HADOOP-5879
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5879
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: io
>            Reporter: Zheng Shao
>         Attachments: hadoop-5879-5-21.patch
>
>
> GzipCodec currently uses the default compression level. We should allow overriding the
default value from Configuration.
> {code}
>   static final class GzipZlibCompressor extends ZlibCompressor {
>     public GzipZlibCompressor() {
>       super(ZlibCompressor.CompressionLevel.DEFAULT_COMPRESSION,
>           ZlibCompressor.CompressionStrategy.DEFAULT_STRATEGY,
>           ZlibCompressor.CompressionHeader.GZIP_FORMAT, 64*1024);
>     }
>   }
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message