hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3540) Further improvement on recovery mode and edit log toleration in branch-1
Date Wed, 26 Sep 2012 01:42:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463455#comment-13463455

Colin Patrick McCabe commented on HDFS-3540:

bq. Hi Colin, thanks for posting a patch. As you mentioned, branch-1 is a stable branch so
that it is better to minimize the change. Let's change the default toleration length to 0.
Then, user could see all the length information before using recovery mode. Does it sound
good to you?

This seems reasonable to me.  

It seems like if we're going to go this route, specifying {{\-recover}} should also cause
{{edit.log.toleration}} to be set to -1 automatically?.  Otherwise Recovery Mode will not
work, and the user will probably be confused as to why.  Perhaps a change like this?

diff --git a/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSImage.java b/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSImage.java
index e4e5045..82d70b4 100644
--- a/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSImage.java
+++ b/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSImage.java
@@ -1031,7 +1031,8 @@ public class FSImage extends Storage {
     int numEdits = 0;
     EditLogFileInputStream edits = 
       new EditLogFileInputStream(getImageFile(sd, NameNodeFile.EDITS));
-    numEdits = FSEditLog.loadFSEdits(edits, editsTolerationLength, recovery);
+    numEdits = FSEditLog.loadFSEdits(edits, 
+        recovery == null ? editsTolerationLength : -1, recovery);
     File editsNew = getImageFile(sd, NameNodeFile.EDITS_NEW);
     if (editsNew.exists() && editsNew.length() > 0) {
> Further improvement on recovery mode and edit log toleration in branch-1
> ------------------------------------------------------------------------
>                 Key: HDFS-3540
>                 URL: https://issues.apache.org/jira/browse/HDFS-3540
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 1.2.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: h3540_20120925.patch, HDFS-3540-b1.004.patch
> *Recovery Mode*: HDFS-3479 backported HDFS-3335 to branch-1.  However, the recovery mode
feature in branch-1 is dramatically different from the recovery mode in trunk since the edit
log implementations in these two branch are different.  For example, there is UNCHECKED_REGION_LENGTH
in branch-1 but not in trunk.
> *Edit Log Toleration*: HDFS-3521 added this feature to branch-1 to remedy UNCHECKED_REGION_LENGTH
and to tolerate edit log corruption.
> There are overlaps between these two features.  We study potential further improvement
in this issue.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message