Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 19 Jan 2017 10:23:26 +0000 (UTC)
From: "SammiChen (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13021357.1479362960000.50868.1484821406631@Atlassian.JIRA>
In-Reply-To: <JIRA.13021357.1479362960000@Atlassian.JIRA>
References: <JIRA.13021357.1479362960000@Atlassian.JIRA> <JIRA.13021357.1479362960944@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-11152) Start erasure coding policy ID
 number from 1 instead of 0 to void potential unexpected errors
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 19 Jan 2017 10:23:33 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829676#comment-15829676 ] 

SammiChen commented on HDFS-11152:
----------------------------------

Hi [~andrew.wang]] I did some investigation. It seems EC policy ID is not widely used.  For examples, 
1. For directory, EC policy is saved as XAttr using policy name(policy name doesn't include policy ID). When directory is saved to FsImage, all its XAttrs are saved including EC policy name.
2. For file, EC policy ID is saved in head. When file is saved to FsImage, EC policy ID is saved in replication field. 
3. The {{setErasureCodingPolicy}} API, user will provide a {{ErasureCodingPolicy}} parameter, currently only the schema and cell size is used to check if it's a legitimate EC policy, policy ID is not involved in parameter checking. Because currently {{setErasureCodingPolicy}} can only applies to directory. And only policy name is saved on directory. I will enforce the policy ID check  in HDFS-10996 implementation which specify per-file policy when create file. 
4. File read/write.  Double checked {{DFSStripedInputStream}} and {{DFSStripedOutputStream}}, policy ID is not used at all. The schema and cellsize are used. 

So, later I plan to upload a new patch, adding Precondition checks on FsImage file loading only.  Because other spaces use policy name, schema and cell size. 


> Start erasure coding policy ID number from 1 instead of 0 to void potential unexpected errors
> ---------------------------------------------------------------------------------------------
>
>                 Key: HDFS-11152
>                 URL: https://issues.apache.org/jira/browse/HDFS-11152
>             Project: Hadoop HDFS
>          Issue Type: Task
>    Affects Versions: 3.0.0-alpha1
>            Reporter: SammiChen
>            Assignee: SammiChen
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HDFS-11152-v1.patch
>
>
> This task will change erasure coding policy ID number starting from 1 instead of current 0, to avoid some potential unexpected errors in codes since 0 is default value for integer variables. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org