Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 15 Jun 2011 01:00:51 +0000 (UTC)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <176628703.5345.1308099651963.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1190453629.5342.1308099527355.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (HDFS-2074) 1073: determine edit log validity by
 truly reading and validating transactions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HDFS-2074:
------------------------------

    Description: 
HDFS-2003 separated the deserialization/reading of log records from the application of those records to a namesystem. This means that we can now read through an edit log in order to determine how many valid transactions are actually stored within. This is an improvement on what the 1073 branch currently does, which is to simply look at how many bytes come before the 0xFFFF... trailer at the end of the file.

The next step after this is to use these new functions so that, when the NN starts up and finds "in-progress" files like "edits_2_inprogress", it will rename them to their finalized name like "edits_2-30" based on how many transactions are truly stored within. This will simplify logic elsewhere.

> 1073: determine edit log validity by truly reading and validating transactions
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-2074
>                 URL: https://issues.apache.org/jira/browse/HDFS-2074
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>
>
> HDFS-2003 separated the deserialization/reading of log records from the application of those records to a namesystem. This means that we can now read through an edit log in order to determine how many valid transactions are actually stored within. This is an improvement on what the 1073 branch currently does, which is to simply look at how many bytes come before the 0xFFFF... trailer at the end of the file.
> The next step after this is to use these new functions so that, when the NN starts up and finds "in-progress" files like "edits_2_inprogress", it will rename them to their finalized name like "edits_2-30" based on how many transactions are truly stored within. This will simplify logic elsewhere.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira