Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <29274329.19251287608427885.JavaMail.jira@thor>
Date: Wed, 20 Oct 2010 17:00:27 -0400 (EDT)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-1073) Simpler model for Namenode's fs Image
 and edit Logs
In-Reply-To: <1207504890.648331270171887350.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923165#action_12923165 ] 

Todd Lipcon commented on HDFS-1073:
-----------------------------------

I'm starting to think about how to convert the current code over to the txid based numbering, came upon a design point I wanted to discuss here:

What should we do about the case when the edits should be rolled, but there have been no transactions since the last roll? For example, consider the following sequence:
1) Start up a fresh NN. We are writing to file edits_0-inprogress
2) Perform 100 edits- now current txid is 100.
3) Perform a roll. This renames edits_0-inprogress to edits_0-100 and opens edits_100-inprogress
4) No more edits, but a new BN starts up, and thus asks for another roll. Thus we would like to create edits_100-100, a file with no edits, which is a little bit strange, and will cause issues the next time we roll (we'll end up with edits_100-100 and also edits_100-200 for example)

It seems the options are:
a) if asked to roll when we have not written any transactions to our current log, it is a no-op
b) whenever we roll, we append a special "trailer" transaction. Thus every log has at least 1 edit in it. I don't really like this, since it means that after a crash, we'll have a log without a trailer, which will add edge cases to worry about.

I'm leaning towards A. Am I missing another good solution?

> Simpler model for Namenode's fs Image and edit Logs 
> ----------------------------------------------------
>
>                 Key: HDFS-1073
>                 URL: https://issues.apache.org/jira/browse/HDFS-1073
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Todd Lipcon
>         Attachments: hdfs-1073.txt, hdfs1073.pdf
>
>
> The naming and handling of  NN's fsImage and edit logs can be significantly improved resulting simpler and more robust code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.