hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11505) Do not enable any erasure coding policies by default
Date Wed, 08 Mar 2017 20:46:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15901943#comment-15901943
] 

Manoj Govindassamy edited comment on HDFS-11505 at 3/8/17 8:46 PM:
-------------------------------------------------------------------

There were a lot of test failures as {{ErasureCodingPolicyManager}} was expecting a valid
policy to passed in during construction. After the following change, I avoided the crash during
the construction and most of the tests started to pass through. Sure, will wait for HDFS-11314
 and post a revised patch with doc fixes.

{code}
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
@@ -98,12 +98,15 @@
         DFSConfigKeys.DFS_NAMENODE_EC_POLICIES_ENABLED_DEFAULT);
     this.enabledPoliciesByName = new TreeMap<>();
     for (String policyName : policyNames) {
+      if (policyName.trim().isEmpty()) {
+        continue;
+      }
{code}


was (Author: manojg):
There were a lot of test failures as {{ErasureCodingPolicyManager}} was expecting a valid
policy to passed in during construction. After the following change, I avoided the crash during
the construction and most of the tests started to pass through. Sure, will wait for HDFS-11314
 and post a revised patch with doc fixes.

+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
@@ -98,12 +98,15 @@
         DFSConfigKeys.DFS_NAMENODE_EC_POLICIES_ENABLED_DEFAULT);
     this.enabledPoliciesByName = new TreeMap<>();
     for (String policyName : policyNames) {
+      if (policyName.trim().isEmpty()) {
+        continue;
+      }

> Do not enable any erasure coding policies by default
> ----------------------------------------------------
>
>                 Key: HDFS-11505
>                 URL: https://issues.apache.org/jira/browse/HDFS-11505
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Andrew Wang
>            Assignee: Manoj Govindassamy
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HDFS-11505.01.patch
>
>
> As discussed on HDFS-11314, administrators need to choose the correct set of EC policies
based on cluster size and desired fault-tolerance properties.
> This means we should not enable any EC policies by default, since any default value could
be incorrect.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message