hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-2874) HA: edit log should log to shared dirs before local dirs
Date Wed, 01 Feb 2012 23:03:53 GMT
HA: edit log should log to shared dirs before local dirs
--------------------------------------------------------

                 Key: HDFS-2874
                 URL: https://issues.apache.org/jira/browse/HDFS-2874
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ha, name-node
    Affects Versions: HA branch (HDFS-1623)
            Reporter: Todd Lipcon
            Priority: Critical


Currently, the NN logs its edits to each of its edits directories in sequence. This can produce
the following bad sequence:
- NN accumulates 100 edits (tx 1-100) in the buffer. Writes and syncs to local drive, then
crashes
- Failover occurs. SBN takes over at txid=1, since txid 1 never got writen.
- First NN restarts. It reads up to txid 100 from its local directories. It is now "ahead"
of the active NN with inconsistent state.
The solution is to write to the shared edits dir, and sync that, before writing to any local
drives.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message