hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode
Date Wed, 15 Apr 2015 01:07:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495498#comment-14495498

Tsz Wo Nicholas Sze commented on HDFS-7859:

> The patch under this JIRA handles saving / loading these default schemas in fsimage.
I think this is necessary even without loading custom schemas from XML. Otherwise we cannot
guarantee the NameNode which loads the fsimage has the same default schemas as the NameNode
which saved it. It is obviously even more necessary when we add custom schemas ...

I think we should not persist anything to NN before we have a clear design since we don't
know what to persist.  For example, should we persist schema ID?  We are not able to answer
this question since we don't even know if a schema should have an ID.

If we change the layout later on, it requires cluster upgrade for the new layout and we have
to support the old layout for backward compatibility.

For now, I suggest to just hard code the only (6,3)-Reed-Solomon schema.  We don't even need
the xml file.

> Erasure Coding: Persist EC schemas in NameNode
> ----------------------------------------------
>                 Key: HDFS-7859
>                 URL: https://issues.apache.org/jira/browse/HDFS-7859
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Xinwei Qin 
>         Attachments: HDFS-7859.001.patch
> In meetup discussion with [~zhz] and [~jingzhao], it's suggested that we persist EC schemas
in NameNode centrally and reliably, so that EC zones can reference them by name efficiently.

This message was sent by Atlassian JIRA

View raw message