hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lufei (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-12088) Remove a user defined EC Policy,the policy is not removed from the userPolicies map
Date Thu, 06 Jul 2017 01:41:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075771#comment-16075771
] 

lufei edited comment on HDFS-12088 at 7/6/17 1:40 AM:
------------------------------------------------------

I'm sorry. It's wrong to removing an ec policy from ECPM for a "remove" operation.It will
leads to the problems you mentioned above.
But the "remove" operation may still has some problems.My operation is as follows:

[root@master lufei]# *hdfs ec -addPolicies -policyFile /home/lufei/hadoop-3.0.0-alpha4-SNAPSHOT/etc/hadoop/user_ec_policies.xml.template*

2017-07-06 09:10:44,007 INFO util.ECPolicyLoader: Loading EC policy file /home/lufei/hadoop-3.0.0-alpha4-SNAPSHOT/etc/hadoop/user_ec_policies.xml.template
Add ErasureCodingPolicy XOR-2-1-128k succeed.
Add ErasureCodingPolicy RS-LEGACY-12-4-256k failed and error message is Codec name RS-legacy
is not supported
[root@master lufei]# *hdfs ec -enablePolicy -policy XOR-2-1-128k*
Erasure coding policy XOR-2-1-128kis enabled
[root@master lufei]# *hdfs ec -listPolicies*
Erasure Coding Policies:
        RS-LEGACY-6-3-64k
        {color:#d04437}XOR-2-1-128k{color}
        XOR-2-1-64k
[root@master lufei]# *hdfs ec -removePolicy -policy XOR-2-1-128k*
Erasure coding policy XOR-2-1-128kis removed
[root@master lufei]# *hdfs ec -listPolicies*
Erasure Coding Policies:
        RS-LEGACY-6-3-64k
        XOR-2-1-64k
[root@master lufei]# *hdfs ec -enablePolicy -policy XOR-2-1-128k*
Erasure coding policy XOR-2-1-128kis enabled
[root@master lufei]# *hdfs ec -listPolicies*
Erasure Coding Policies:
        RS-LEGACY-6-3-64k
        {color:#d04437}XOR-2-1-128k{color}
        XOR-2-1-64k

As my operations shown above, I am confused by the current implementation too. 
As the current implementation,a "remove" and a "disable" have the same effect.
And the removed EC policy should not be able to recover by a "enable" operate, but only can
recover by a "add" operate.



was (Author: figo):
I'm sorry. It's wrong to removing an ec policy from ECPM for a "remove" operation.It will
leads to the problems you mentioned above.
But the "remove" operation may still has some problems.My operation is as follows:

+[root@master lufei]# *hdfs ec -addPolicies -policyFile /home/lufei/hadoop-3.0.0-alpha4-SNAPSHOT/etc/hadoop/user_ec_policies.xml.template*

2017-07-06 09:10:44,007 INFO util.ECPolicyLoader: Loading EC policy file /home/lufei/hadoop-3.0.0-alpha4-SNAPSHOT/etc/hadoop/user_ec_policies.xml.template
Add ErasureCodingPolicy XOR-2-1-128k succeed.
Add ErasureCodingPolicy RS-LEGACY-12-4-256k failed and error message is Codec name RS-legacy
is not supported
[root@master lufei]# *hdfs ec -enablePolicy -policy XOR-2-1-128k*
Erasure coding policy XOR-2-1-128kis enabled
[root@master lufei]# *hdfs ec -listPolicies*
Erasure Coding Policies:
        RS-LEGACY-6-3-64k
        {color:#d04437}XOR-2-1-128k{color}
        XOR-2-1-64k
[root@master lufei]# *hdfs ec -removePolicy -policy XOR-2-1-128k*
Erasure coding policy XOR-2-1-128kis removed
[root@master lufei]# *hdfs ec -listPolicies*
Erasure Coding Policies:
        RS-LEGACY-6-3-64k
        XOR-2-1-64k
[root@master lufei]# *hdfs ec -enablePolicy -policy XOR-2-1-128k*
Erasure coding policy XOR-2-1-128kis enabled
[root@master lufei]# *hdfs ec -listPolicies*
Erasure Coding Policies:
        RS-LEGACY-6-3-64k
        {color:#d04437}XOR-2-1-128k{color}
        XOR-2-1-64k+

As my operations shown above, I am confused by the current implementation too. 
As the current implementation,a "remove" and a "disable" have the same effect.
And the removed EC policy should not be able to recover by a "enable" operate, but only can
recover by a "add" operate.


> Remove a user defined EC Policy,the policy is not removed from the userPolicies map
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-12088
>                 URL: https://issues.apache.org/jira/browse/HDFS-12088
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding, hdfs
>    Affects Versions: 3.0.0-alpha3
>            Reporter: lufei
>            Assignee: lufei
>            Priority: Critical
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HDFS-12088.001.patch
>
>
> When user remove an user defined EC policy, it needs to remove the policy from the userPolicies
Map but not only remove from the enabledPolicies Map.Otherwise, after remove the user defined
EC policy, user can recover the EC policy by enable the same EC policy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message