hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones
Date Wed, 05 Aug 2015 23:06:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659150#comment-14659150
] 

Jing Zhao commented on HDFS-8833:
---------------------------------

I understand this, but still thanks for the explanation, Andrew. I'm not opposing to support
multiple EC schemas. My point is we should only use xattr for representing ec policy for file/dir,
instead of occupying several bits in INodeFile first then coming to xattr later. 

Also, instead of doing this extension at this stage, I think it's better to support one schema
only first, and spend more time on testing and stabilizing the current implementation, especially
considering we still have some issues with the write code path.

> Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC
zones
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8833
>                 URL: https://issues.apache.org/jira/browse/HDFS-8833
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-7285
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>
> We have [discussed | https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14357754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14357754]
storing EC schema with files instead of EC zones and recently revisited the discussion under
HDFS-8059.
> As a recap, the _zone_ concept has severe limitations including renaming and nested configuration.
Those limitations are valid in encryption for security reasons and it doesn't make sense to
carry them over in EC.
> This JIRA aims to store EC schema and cell size on {{INodeFile}} level. For simplicity,
we should first implement it as an xattr and consider memory optimizations (such as moving
it to file header) as a follow-on. We should also disable changing EC policy on a non-empty
file / dir in the first phase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message