hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-3797) QJM: add segment txid as a parameter to journal() RPC
Date Mon, 13 Aug 2012 22:42:38 GMT
Todd Lipcon created HDFS-3797:

             Summary: QJM: add segment txid as a parameter to journal() RPC
                 Key: HDFS-3797
                 URL: https://issues.apache.org/jira/browse/HDFS-3797
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ha
    Affects Versions: QuorumJournalManager (HDFS-3077)
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon
            Priority: Minor

During fault testing of QJM, I saw the following issue:

1) NN sends txn 5 to JN
2) NN gets partitioned from JN while JN remains up. The next two RPCs are missed while the
partition has happened:
2a) finalizeSegment(1-5)
2b) startSegment(6)
3) NN sends txn 6 to JN

This caused one of the JNs to end up with a segment 1-10 while the others had two segments;
1-5 and 6-10. This broke some invariants of the QJM protocol and prevented the recovery protocol
from running properly.

This can be addressed on the client side by HDFS-3726, which would cause the NN to not send
the RPC in #3. But it makes sense to also add an extra safety check here on the server side:
with every journal() call, we can send the segment's txid. Then if the JN and the client get
"out of sync", the JN can reject the RPCs.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message