hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them
Date Wed, 10 Oct 2012 17:49:03 GMT
Todd Lipcon created HDFS-4025:
---------------------------------

             Summary: QJM: Sychronize past log segments to JNs that missed them
                 Key: HDFS-4025
                 URL: https://issues.apache.org/jira/browse/HDFS-4025
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ha
    Affects Versions: QuorumJournalManager (HDFS-3077)
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon


Currently, if a JournalManager crashes and misses some segment of logs, and then comes back,
it will be re-added as a valid part of the quorum on the next log roll. However, it will not
have a complete history of log segments (i.e any individual JN may have gaps in its transaction
history). This mirrors the behavior of the NameNode when there are multiple local directories
specified.

However, it would be better if a background thread noticed these gaps and "filled them in"
by grabbing the segments from other JournalNodes. This increases the resilience of the system
when JournalNodes get reformatted or otherwise lose their local disk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message