hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-6801) io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java
Date Sat, 25 Jun 2011 12:22:47 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J updated HADOOP-6801:
----------------------------

    Attachment: HADOOP-6801.r1.diff

Patch that introduces two new options over io.sort.mb and io.sort.factor, specifically for
SequenceFile.Sorter:

{code}
seq.io.sort.mb = 100
seq.io.sort.factor = 100
{code}

Descriptions included inside {{core-default.xml}} changes as well.

I did not yet write checks for _deprecated_ io.sort.* properties. Is the double deprecation
fine here (MR already deprecates this, and now Common has to too, for new SequenceFile.Sorter
specific properties)?

I believe DistCp would need documentation work as well, since it uses SequenceFile.Sorter

> io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java
and used in SequenceFile.java
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6801
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 0.22.0
>            Reporter: Erik Steffl
>            Assignee: Harsh J
>            Priority: Minor
>         Attachments: HADOOP-6801.r1.diff
>
>
> Following configuration keys in CommonConfigurationKeysPublic.java (former CommonConfigurationKeys.java):
> public static final String  IO_SORT_MB_KEY = "io.sort.mb";
> public static final String  IO_SORT_FACTOR_KEY = "io.sort.factor";
> are partially moved:
>   - they were renamed to mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor
respectively
>   - they were moved to mapreduce project, documented in mapred-default.xml
> However:
>   - they are still listed in CommonConfigurationKeysPublic.java as quoted above
>   - strings "io.sort.mb" and "io.sort.factor" are used in SequenceFile.java in Hadoop
Common project
> Not sure what the solution is, these constants should probably be removed from CommonConfigurationKeysPublic.java
but I am not sure what's the best solution for SequenceFile.java.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message