hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version
Date Wed, 15 Jan 2014 23:16:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872771#comment-13872771
] 

Colin Patrick McCabe commented on HDFS-5223:
--------------------------------------------

I think the way to do backwards compatible feature flags would be to have a flag prefix such
as "compat_".  This would let older NameNodes know that the (new, unknown to them) flag they
were looking at was compatible.  The information needs to be in the flag name, since it can't
be in the older software.

However, I don't think we should implement this now.  We don't really have any infrastructure
in place today to use "backwards compatible feature flags."  With the simple DataInputStream/DataOutputStream
based decoding we have now, any extra field results in a loading failure.

So it's fair to say: we can't create even one "backwards compatible feature flag" without
installing new software beyond what this patch provides.  Given that this is true, we should
just implement compatible feature flags later when we know we need (or at least can use) them.
 I am all for doing it after the protobuf merge.  But there's simply no reason to do it before
because I don't believe a backwards compatible change can be made at this point.

> Allow edit log/fsimage format changes without changing layout version
> ---------------------------------------------------------------------
>
>                 Key: HDFS-5223
>                 URL: https://issues.apache.org/jira/browse/HDFS-5223
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.1.1-beta
>            Reporter: Aaron T. Myers
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-5223.004.patch
>
>
> Currently all HDFS on-disk formats are version by the single layout version. This means
that even for changes which might be backward compatible, like the addition of a new edit
log op code, we must go through the full `namenode -upgrade' process which requires coordination
with DNs, etc. HDFS should support a lighter weight alternative.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message