hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14679) failed to add erasure code policies with example template
Date Wed, 31 Jul 2019 03:55:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896756#comment-16896756
] 

Hadoop QA commented on HDFS-14679:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 14s{color} | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  0s{color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 56s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  9s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 38m 34s{color}
| {color:green} branch has no errors when building and testing our client artifacts. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m  2s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  5s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 36s{color}
| {color:green} patch has no errors when building and testing our client artifacts. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m  0s{color} | {color:green}
hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 26s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 58s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HDFS-14679 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12976284/HDFS-14679-02.patch
|
| Optional Tests |  dupname  asflicense  mvnsite  unit  |
| uname | Linux 7bff0e64e3e5 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0f2dad6 |
| maven | version: Apache Maven 3.3.9 |
|  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/27341/testReport/ |
| Max. process+thread count | 306 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/27341/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> failed to add erasure code policies with example template
> ---------------------------------------------------------
>
>                 Key: HDFS-14679
>                 URL: https://issues.apache.org/jira/browse/HDFS-14679
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: ec
>    Affects Versions: 3.1.2
>            Reporter: Yuan Zhou
>            Assignee: Yuan Zhou
>            Priority: Minor
>         Attachments: HDFS-14679-01.patch, HDFS-14679-02.patch, fix_adding_EC_policy_example.diff
>
>
> Hi Hadoop developers,
>  
> Trying to do some quick tests with erasure coding feature and ran into a issue on adding
policies. The example on adding erasure code policies with provided template failed:
> {quote}./bin/hdfs ec -addPolicies -policyFile /tmp/user_ec_policies.xml.template
>  2019-07-30 10:35:16,447 INFO util.ECPolicyLoader: Loading EC policy file /tmp/user_ec_policies.xml.template
>  Add ErasureCodingPolicy XOR-2-1-128k succeed.
>  Add ErasureCodingPolicy RS-LEGACY-12-4-256k failed and error message is Codec name RS-legacy
is not supported
> {quote}
> The issue seems due to be the mismatching codec(upper case vs lower case). The codec
is in upper case in the example template[1] while all available codecs are lower case[2].
A way to fix maybe just converting the codec to lower case when parsing the policy schema.
Also attached a simple patch here. 
> [1] [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template#L51]
> [2][https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/ErasureCodeConstants.java#L28-L33]
> Thanks, -yuan



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message