hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "BELUGA BEHR (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-14668) Remove Configurable Default Sequence File Compression Type
Date Tue, 18 Jul 2017 18:57:00 GMT
BELUGA BEHR created HADOOP-14668:

             Summary: Remove Configurable Default Sequence File Compression Type
                 Key: HADOOP-14668
                 URL: https://issues.apache.org/jira/browse/HADOOP-14668
             Project: Hadoop Common
          Issue Type: Improvement
          Components: io
    Affects Versions: 3.0.0-alpha3
            Reporter: BELUGA BEHR
            Priority: Trivial
             Fix For: 2.8.1

It is confusing to have two different ways to set the Sequence File compression type.

In a basic configuration, I can set _mapreduce.output.fileoutputformat.compress.type_ or _io.seqfile.compression.type_.
 If I would like to set a default value, I should set it by setting the cluster environment's
mapred-site.xml file setting for _mapreduce.output.fileoutputformat.compress.type_.

Please remove references to this magic string _io.seqfile.compression.type_, remove the {{setDefaultCompressionType}}
method, and have {{getDefaultCompressionType}} return value hard-coded to {{CompressionType.RECORD}}.
 This will make administration easier as I have to only interrogate one configuration.

   * Get the compression type for the reduce outputs
   * @param job the job config to look in
   * @return the kind of compression to use
  static public CompressionType getDefaultCompressionType(Configuration job) {
    String name = job.get("io.seqfile.compression.type");
    return name == null ? CompressionType.RECORD : 
   * Set the default compression type for sequence files.
   * @param job the configuration to modify
   * @param val the new compression type (none, block, record)
  static public void setDefaultCompressionType(Configuration job, 
                                               CompressionType val) {
    job.set("io.seqfile.compression.type", val.toString());

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message