Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Date: Thu, 16 Aug 2012 06:15:38 +1100 (NCT)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-dev@hadoop.apache.org
Message-ID: <223216776.15413.1345058138195.JavaMail.jiratomcat@arcas>
In-Reply-To: <605892790.4766.1344897758079.JavaMail.jiratomcat@arcas>
Subject: [jira] [Resolved] (HDFS-3797) QJM: add segment txid as a parameter
 to journal() RPC
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon resolved HDFS-3797.
-------------------------------

       Resolution: Fixed
    Fix Version/s: QuorumJournalManager (HDFS-3077)
     Hadoop Flags: Reviewed
    
> QJM: add segment txid as a parameter to journal() RPC
> -----------------------------------------------------
>
>                 Key: HDFS-3797
>                 URL: https://issues.apache.org/jira/browse/HDFS-3797
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha
>    Affects Versions: QuorumJournalManager (HDFS-3077)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>             Fix For: QuorumJournalManager (HDFS-3077)
>
>         Attachments: hdfs-3797.txt
>
>
> During fault testing of QJM, I saw the following issue:
> 1) NN sends txn 5 to JN
> 2) NN gets partitioned from JN while JN remains up. The next two RPCs are missed while the partition has happened:
> 2a) finalizeSegment(1-5)
> 2b) startSegment(6)
> 3) NN sends txn 6 to JN
> This caused one of the JNs to end up with a segment 1-10 while the others had two segments; 1-5 and 6-10. This broke some invariants of the QJM protocol and prevented the recovery protocol from running properly.
> This can be addressed on the client side by HDFS-3726, which would cause the NN to not send the RPC in #3. But it makes sense to also add an extra safety check here on the server side: with every journal() call, we can send the segment's txid. Then if the JN and the client get "out of sync", the JN can reject the RPCs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira