hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3947) If there are multiple edit logs with the same fstime, Recovery Mode should load a different one than the normal loading process
Date Mon, 11 Mar 2013 18:37:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599106#comment-13599106
] 

Colin Patrick McCabe commented on HDFS-3947:
--------------------------------------------

I talked to Aaron T. Meyers about this and he pointed out a flaw: since we write edit logs
one after another, they won't have exactly the same creation time.  So it's not as easy as
just inverting the order of edit log reading for the files with the latest timestamp.

We could invert the order regardless of timestamp, or read the second-most-recent log, but
that starts getting a little bit complex.  Since this is fixed in trunk (by {{RedundantEditLogInputStream}}),
I think it might be better to close this, unless someone wants to pick it up again.
                
> If there are multiple edit logs with the same fstime, Recovery Mode should load a different
one than the normal loading process
> -------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3947
>                 URL: https://issues.apache.org/jira/browse/HDFS-3947
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 1.2.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HDFS-3947-b1.001.patch
>
>
> If there are multiple edit logs with the same {{fstime}}, Recovery Mode should load a
different one than the normal loading process.  This will protect against many common kinds
of corruption which affect only one edit log directory.  For example, most I/O errors would
fall into this category.
> This improvement is not necessary in branch-2 and later branches because in those branches,
we have edit log failover (See HDFS-3049).  However, this is an extremely simple and useful
thing we can do to make Recovery Mode more helpful in branch-1.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message