Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 1 May 2012 20:26:53 +0000 (UTC)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1568826791.14383.1335904013960.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <437643778.4197.1335558050636.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-3335) check for edit log corruption at the
 end of the log
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266040#comment-13266040 ] 

Todd Lipcon commented on HDFS-3335:
-----------------------------------

A few thoughts:
- we need to do this in trunk before we can review/commit to branch-1
- I'm afraid this may break the HA use case, where the previous active may crash in the middle of an fsync(). This could leave a partial edit at the end of the file. Let's complete HDFS-2766 before we enable this lest it break the HA case.
                
> check for edit log corruption at the end of the log
> ---------------------------------------------------
>
>                 Key: HDFS-3335
>                 URL: https://issues.apache.org/jira/browse/HDFS-3335
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 0.23.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-3335-b1.001.patch, HDFS-3335-b1.002.patch
>
>
> Even after encountering an OP_INVALID, we should check the end of the edit log to make sure that it contains no more edits.
> This will catch things like rare race conditions or log corruptions that would otherwise remain undetected.  They will got from being silent data loss scenarios to being cases that we can detect and fix.
> Using recovery mode, we can choose to ignore the end of the log if necessary.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira