hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones
Date Wed, 02 Sep 2015 06:56:45 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Zhe Zhang updated HDFS-8833:
    Attachment: HDFS-8833-HDFS-7285.05.patch

Thanks Rakesh for finding the issue! Updating the patch to remove all references to "EC zone".
I've also just updated the main HDFS-7285 feature branch to sync with trunk.

The above summary looks good, thanks Rakesh for the idea. I will post an updated list soon.

bq. dir already has a policy -> not allows to set EC policy again to this dir
This is related to the nested policy discussion. Since we only support 1 policy now, the above
is true -- it doesn't make sense to reset the same policy. We have agreed to revisit the nested
policy design when we support multiple policies.

Since the patch is already large, we should probably address the documentation updates in
HDFS-7351, including the {{setPolicy}} example that Jing suggested.

> Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC
> -------------------------------------------------------------------------------------------
>                 Key: HDFS-8833
>                 URL: https://issues.apache.org/jira/browse/HDFS-8833
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-7285
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8833-HDFS-7285-merge.00.patch, HDFS-8833-HDFS-7285-merge.01.patch,
HDFS-8833-HDFS-7285.02.patch, HDFS-8833-HDFS-7285.03.patch, HDFS-8833-HDFS-7285.04.patch,
> We have [discussed | https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14357754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14357754]
storing EC schema with files instead of EC zones and recently revisited the discussion under
> As a recap, the _zone_ concept has severe limitations including renaming and nested configuration.
Those limitations are valid in encryption for security reasons and it doesn't make sense to
carry them over in EC.
> This JIRA aims to store EC schema and cell size on {{INodeFile}} level. For simplicity,
we should first implement it as an xattr and consider memory optimizations (such as moving
it to file header) as a follow-on. We should also disable changing EC policy on a non-empty
file / dir in the first phase.

This message was sent by Atlassian JIRA

View raw message