hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1073) Simpler model for Namenode's fs Image and edit Logs
Date Fri, 05 Nov 2010 16:42:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928672#action_12928672
] 

Sanjay Radia commented on HDFS-1073:
------------------------------------

>... but then you get the problem of synchronizing the start of a checkpoint and the edits
roll event. Otherwise checkpoints may become way behind the current namespace state.
I guess I am missing this. 
We should avoid the synchronization that has been there in the original design of the secondary
NN.
The BN can checkpoint whenever it feels that the set of rolled edits since previous checkpoint
is large enough. It may be simpler to do it on every roll if we have configured the
NN to roll say every 10K transactions.
Perhaps what I am proposing works for the checkpointer but not for the BN because of some
property of the BN that I am missing.

> Simpler model for Namenode's fs Image and edit Logs 
> ----------------------------------------------------
>
>                 Key: HDFS-1073
>                 URL: https://issues.apache.org/jira/browse/HDFS-1073
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Todd Lipcon
>         Attachments: hdfs-1073.txt, hdfs1073.pdf
>
>
> The naming and handling of  NN's fsImage and edit logs can be significantly improved
resulting simpler and more robust code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message