hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ayush Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14679) Failed to add erasure code policies with example template
Date Tue, 06 Aug 2019 03:30:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900576#comment-16900576

Ayush Saxena commented on HDFS-14679:

Committed to trunk.
Thanx [~yuanzhou] for the contribution!!!

> Failed to add erasure code policies with example template
> ---------------------------------------------------------
>                 Key: HDFS-14679
>                 URL: https://issues.apache.org/jira/browse/HDFS-14679
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: ec
>    Affects Versions: 3.1.2
>            Reporter: Yuan Zhou
>            Assignee: Yuan Zhou
>            Priority: Minor
>         Attachments: HDFS-14679-01.patch, HDFS-14679-02.patch, HDFS-14679-03.patch, fix_adding_EC_policy_example.diff
> Hi Hadoop developers,
> Trying to do some quick tests with erasure coding feature and ran into a issue on adding
policies. The example on adding erasure code policies with provided template failed:
> {quote}./bin/hdfs ec -addPolicies -policyFile /tmp/user_ec_policies.xml.template
>  2019-07-30 10:35:16,447 INFO util.ECPolicyLoader: Loading EC policy file /tmp/user_ec_policies.xml.template
>  Add ErasureCodingPolicy XOR-2-1-128k succeed.
>  Add ErasureCodingPolicy RS-LEGACY-12-4-256k failed and error message is Codec name RS-legacy
is not supported
> {quote}
> The issue seems due to be the mismatching codec(upper case vs lower case). The codec
is in upper case in the example template[1] while all available codecs are lower case[2].
A way to fix maybe just converting the codec to lower case when parsing the policy schema.
Also attached a simple patch here. 
> [1] [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template#L51]
> [2][https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ErasureCodeConstants.java#L28-L33]
> Thanks, -yuan

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message