Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Thu, 30 Jul 2015 18:41:05 +0000 (UTC)
From: "Andrew Wang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12849790.1438125336000.333173.1438281665383@Atlassian.JIRA>
In-Reply-To: <JIRA.12849790.1438125336000@Atlassian.JIRA>
References: <JIRA.12849790.1438125336000@Atlassian.JIRA>
 <JIRA.12849790.1438125336128@arcas>
Subject: [jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and
 cell size with INodeFile and eliminate EC zones
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648107#comment-14648107 ] 

Andrew Wang commented on HDFS-8833:
-----------------------------------

bq. StoragePolicy do not have such restriction since moving a WARM file into a COLD dir as well changes its underlining storage (mover will move the replicas in order to satisfy the policy.

IIUC this only happens if the file is set to "inherit" and picked up WARM from a parent directory. If the storage policy was set on the file itself, it would not change when renamed under a COLD directory.

The proposal is basically like storage policies, except there is no "inherit" mode.

> Erasure coding: store EC schema and cell size with INodeFile and eliminate EC zones
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-8833
>                 URL: https://issues.apache.org/jira/browse/HDFS-8833
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-7285
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>
> We have [discussed | https://issues.apache.org/jira/browse/HDFS-7285?focusedCommentId=14357754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14357754] storing EC schema with files instead of EC zones and recently revisited the discussion under HDFS-8059.
> As a recap, the _zone_ concept has severe limitations including renaming and nested configuration. Those limitations are valid in encryption for security reasons and it doesn't make sense to carry them over in EC.
> This JIRA aims to store EC schema and cell size on {{INodeFile}} level. For simplicity, we should first implement it as an xattr and consider memory optimizations (such as moving it to file header) as a follow-on. We should also disable changing EC policy on a non-empty file / dir in the first phase.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)