hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3725) Fix QJM startup when individual JNs have gaps
Date Mon, 30 Jul 2012 23:12:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425357#comment-13425357
] 

Todd Lipcon commented on HDFS-3725:
-----------------------------------

I think this is safe, because we do the same checking for gaps on the client side while we're
loading. On loading each transaction, we verify that it's the "expected" one (i.e the previous
transaction + 1).
                
> Fix QJM startup when individual JNs have gaps
> ---------------------------------------------
>
>                 Key: HDFS-3725
>                 URL: https://issues.apache.org/jira/browse/HDFS-3725
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha
>    Affects Versions: QuorumJournalManager (HDFS-3077)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>         Attachments: hdfs-3725.txt
>
>
> This fixes a bug in the HDFS-3077 branch:
> If one of the JNs goes down and comes back up, it may have a discontiguous set of logs
in its storage directories. This means that a future {{getEditLogManifest}} call will return
a manifest that has gaps - eg it may have txn 1-3 and 7-9 but be missing 4-6. This is OK because
it's guaranteed that a quorum of JNs do have those edits elsewhere.
> This situation currently causes an exception since the RemoteEditLogManifest constructor
verifies that the returned log files form a contiguous set of edits. That was true for the
NameNode but no longer true for this new usage.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message