hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3540) Further improvement on recovery mode and edit log toleration in branch-1
Date Tue, 04 Sep 2012 18:31:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447906#comment-13447906

Colin Patrick McCabe commented on HDFS-3540:

bq. Correct me if I am wrong: Recovery Mode without HDFS-3479 means the entire end-of-log
is not checked and, therefore, the silent data loss length is not limited. It is even worst.

No, that is incorrect.

Recovery mode has always read up to the end of the log, and it always will.  The confusion
arises because sometimes we are not very good at determining where the "end of the log" is.

I filed and implemented HDFS-3479 because I noticed that in certain scenarios we would decide
that the edit log ended before it really did because we spotted an {{OP_INVALID}}.

The unchecked region which we've been discussing only applied to HDFS-3479 corruption, not
to any other type of corruption.  Frankly, the unchecked region was a mistake.

However, none of this has *anything* to do with recovery mode.  HDFS-3004 and HDFS-3479 were
separate JIRAs, that implemented separate features.
> Further improvement on recovery mode and edit log toleration in branch-1
> ------------------------------------------------------------------------
>                 Key: HDFS-3540
>                 URL: https://issues.apache.org/jira/browse/HDFS-3540
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 1.2.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
> *Recovery Mode*: HDFS-3479 backported HDFS-3335 to branch-1.  However, the recovery mode
feature in branch-1 is dramatically different from the recovery mode in trunk since the edit
log implementations in these two branch are different.  For example, there is UNCHECKED_REGION_LENGTH
in branch-1 but not in trunk.
> *Edit Log Toleration*: HDFS-3521 added this feature to branch-1 to remedy UNCHECKED_REGION_LENGTH
and to tolerate edit log corruption.
> There are overlaps between these two features.  We study potential further improvement
in this issue.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message