hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11975) Provide a system-default EC policy
Date Mon, 31 Jul 2017 02:26:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106737#comment-16106737

SammiChen commented on HDFS-11975:

Hi [~luhuichun],   I have several suggestions,
1.  {{setErasureCodingPolicy}} in NameNodeRpcServer.java, suggest lower the log level to debug
or trace. Log level "info" will produce too much daily logs 
2. new test added into testErasureCodingConf.xml, after set the default ec policy, it's better
to get the ec policy on the directory, to make sure it's the expected ec policy
3. except testErasureCodingConf.xml,  it's also recommended to add unit test in {{TestErasureCodingPolicies}}
4. the default ec policy should register itself to {{ErasureCodingPolicyManager}}, so that
it will be enabled from {{ErasureCodingPolicyManager}} point of view

> Provide a system-default EC policy
> ----------------------------------
>                 Key: HDFS-11975
>                 URL: https://issues.apache.org/jira/browse/HDFS-11975
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Lei (Eddy) Xu
>            Assignee: luhuichun
>         Attachments: HDFS-11975-001.patch, HDFS-11975-002.patch, HDFS-11975-003.patch,
HDFS-11975-004.patch, HDFS-11975-005.patch, HDFS-11975-006.patch, HDFS-11975-007.patch, HDFS-11975-008.patch
> From the usability point of view, it'd be nice to be able to specify a system-wide EC
policy, i.e., in {{hdfs-site.xml}}. For most of users / admin / downstream projects, it is
not necessary to know the tradeoffs of the EC policy, considering that it requires the knowledge
of EC, the actual physical topology of the clusters, and many other factors (i.e., network,
cluster size and etc).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message