hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5879) GzipCodec should read compression level etc from configuration
Date Tue, 26 May 2009 06:30:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712875#action_12712875
] 

He Yongqiang commented on HADOOP-5879:
--------------------------------------

I checked the code, it will work with SequenceFile, for example:
{noformat}
  public static void testSequenceFileWithGZipCodec() throws IOException{
    Configuration confWithParam = new Configuration();
    confWithParam.set("io.compress.level", "1");
    confWithParam.set("io.compress.strategy", "0");
    confWithParam.set("io.compress.buffer.size", Integer.toString(128 * 1024));
    FileSystem fs =FileSystem.get(confWithParam);
    GzipCodec codec= new GzipCodec();
    codec.setConf(confWithParam);// this line is not needed for creating SequenceFile.Writer
    SequenceFile.Writer writer =SequenceFile.createWriter(fs, confWithParam, new Path("/test/path"),

       NullWritable.class, NullWritable.class, 
        CompressionType.BLOCK, codec);
    writer.close();
  }
{noforamt}
The call trace for getting a compressor is :
CodecPool.getCompressor(CompressionCodec)-->GzipCodec.createCompressor()-->new GzipZlibCompressor(conf).
I have not checked IFile, but i think it should work in the same way.

> GzipCodec should read compression level etc from configuration
> --------------------------------------------------------------
>
>                 Key: HADOOP-5879
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5879
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: io
>            Reporter: Zheng Shao
>         Attachments: hadoop-5879-5-21.patch
>
>
> GzipCodec currently uses the default compression level. We should allow overriding the
default value from Configuration.
> {code}
>   static final class GzipZlibCompressor extends ZlibCompressor {
>     public GzipZlibCompressor() {
>       super(ZlibCompressor.CompressionLevel.DEFAULT_COMPRESSION,
>           ZlibCompressor.CompressionStrategy.DEFAULT_STRATEGY,
>           ZlibCompressor.CompressionHeader.GZIP_FORMAT, 64*1024);
>     }
>   }
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message