hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version
Date Thu, 26 Sep 2013 20:22:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779177#comment-13779177

Nathan Roberts commented on HDFS-5223:

Thanks Aaron and Todd for bringing this up.

I love the flexibility of feature bits however I'm very nervous about the complexity they
tend to bring. As long as there is incredibly tight controls it can work but more often than
not I've seen this sort of approach lead to some incredibly unmaintainable code. The code
can get very complex dealing with multiple combinations and the testing/QA can get also be
very difficult to manage. Things can get overwhelmingly complex quite quickly. Having an "-enableAllNewFeatures"
helps a bit but I'm not sure it lowers the complexity all that much. 

Of the two options, I'd lean in the direction of #1 at this point. 

iiuc, option 2 basically means that V2 software has to remember how to both read and write
in V1 format whereas option 1 only requires that V2 be able to read V1 format (like we do
today). I kind of like the fact that new software doesn't ever have to write things according
to the older format. 

* When we update the SBN to V2 it would be allowed to come up and it would still be able to
process V1 images/edits
* The first time it tries to write a new image, it would do so in V2 format 
* When uploading a new V2 image to ANN, the upload would not proceed because of the version
mismatch (this way the ANN's local storage stays purely V1)
* At this point we can still rollback by simply re-bootstrapping the SBN
* Now we failover to the SBN, the SBN changes the shared edits area to indicate V2 (just an
update to VERSION file I think)
* Upgrade old ANN with V2 software
* old ANN comes up as Standby, reads the new V2 image and starts processing new V2 edits (somewhere
in here also has to change local storage to V2)

What's not great about this approach is that as soon as V2 software becomes active, we're
writing in V2 format and at that point can't go back without losing edits. However, that's
basically very similar to today's -upgrade. The only difference being that we haven't done
anything to protect the blocks on the datanodes (with -upgrade we hardlink everything and
therefore guarantee data blocks can't go away). So, maybe we need a mode where HDFS stops
deleting blocks both from the NN's perspective (won't issue invalidates any longer), as well
as from the DN side where it will ignore block deletion requests. Kind of a semi-safe-mode
where the filesystem acts pretty much normally except that it refuses to delete any blocks.
If we get ourselves into a true disaster-recovery situation, we can go back to V1 software
+ last V1 fsimage + all V1 edits that applied to that image + all blocks from the datanodes.

> Allow edit log/fsimage format changes without changing layout version
> ---------------------------------------------------------------------
>                 Key: HDFS-5223
>                 URL: https://issues.apache.org/jira/browse/HDFS-5223
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.1.1-beta
>            Reporter: Aaron T. Myers
> Currently all HDFS on-disk formats are version by the single layout version. This means
that even for changes which might be backward compatible, like the addition of a new edit
log op code, we must go through the full `namenode -upgrade' process which requires coordination
with DNs, etc. HDFS should support a lighter weight alternative.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message