hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
Date Mon, 05 Apr 2010 19:02:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853490#action_12853490

Sanjay Radia commented on HDFS-1073:

> Sanjay mentioned "it decouples the split (ie roll) of the edit log and the checkpoint
of the image" but I'm not sure what he meant by that. 
>I think we can still achieve the same goal using indexed files, ...  fsimage_0, edits_0,
edits_1, edits_2, fsimage_2

The serial numbering of the files solution requires that  checkpoints  occur only at a edits
split boundaries.
The transaction ID one does not have that restriction but it does require that in order to
detect a gap in edits one has to look inside the logs. The txId one can avoid that if we are
prepared to rename the edits log when you split (roll) it (Ugh!)
The txId numbering scheme also has the advantage that multiple backups can roll and do checkpoints
independently (we *DONOT* want to do that as it will confuse the operators -- but it shows
that the design is very robust.

The options to have any server to a checkpoint is useful. I think both schemes allow that.

> Simpler model for Namenode's fs Image and edit Logs 
> ----------------------------------------------------
>                 Key: HDFS-1073
>                 URL: https://issues.apache.org/jira/browse/HDFS-1073
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Todd Lipcon
> The naming and handling of  NN's fsImage and edit logs can be significantly improved
resulting simpler and more robust code.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message