hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
Date Thu, 05 May 2011 20:39:04 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated HDFS-1073:

    Attachment: hdfs1073.pdf

Updated version of the design doc based on feedback from Eli and Sanjay.

A few responses to your points:

bq. Section 1.2, is the new OP_INVALID filler relevant here since a journal will also contain
these txns in practice?

I don't think it's relevant to this document, since it has always existed. The recent change
was just that we fill with OP_INVALID all the way to the end of the file, whereas before it
just used to be a single OP_INVALID followed by 0x00s.

bq. Section 4.5. For your open question, I don't think we should support upgrade from a namespace
that was not cleanly shut down. Ie let's restrict the space of logs an upgrade needs to deal
with, the admin start and cleanly shutdown the Namenode before upgrading, which seems reasonable
to require, and should be the common case anyway.

Actually, in the way this branch has progressed, we're maintaining today's behavior of supporting
unclean upgrade just fine. We should of course QA it, but it didn't turn out to be particular
complex after refactoring the startup process.

> Simpler model for Namenode's fs Image and edit Logs 
> ----------------------------------------------------
>                 Key: HDFS-1073
>                 URL: https://issues.apache.org/jira/browse/HDFS-1073
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Todd Lipcon
>         Attachments: hdfs-1073-editloading-algos.txt, hdfs-1073.txt, hdfs1073.pdf, hdfs1073.pdf,
> The naming and handling of  NN's fsImage and edit logs can be significantly improved
resulting simpler and more robust code.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message