Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 2 Apr 2012 17:45:30 +0000 (UTC)
From: "Todd Lipcon (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <892448245.1494.1333388730417.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <595597907.12772.1331732678494.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-3092) Enable journal protocol based
 editlog streaming for standby namenode
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244369#comment-13244369 ] 

Todd Lipcon commented on HDFS-3092:
-----------------------------------

bq. For the scenario you described, NN2 after it rolls JD2 and JD3, updates the JournalList with JD2 and JD3. JD1 will no longer be used.

The situation here is at the beginning of a segment - for example the very first transaction. So, when NN2 rolls, the starting txid of the next segment is 1. I think you need to add an epoch number which is separate from the txid, to distinguish different "startings" of the same segment.

bq. Fence command will include a version number that we got from the JournalList ZK node. The number that is higher wins at the JD. The fence command with lower version # is rejected.

You'll also need to atomically write this to local storage.


I'll upload my in-progress code to HDFS-3077 so that we might be able to start sharing code earlier rather than later.
                
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>
>                 Key: HDFS-3092
>                 URL: https://issues.apache.org/jira/browse/HDFS-3092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: MultipleSharedJournals.pdf
>
>
> Currently standby namenode relies on reading shared editlogs to stay current with the active namenode, for namespace changes. BackupNode used streaming edits from active namenode for doing the same. This jira is to explore using journal protocol based editlog streams for the standby namenode. A daemon in standby will get the editlogs from the active and write it to local edits. To begin with, the existing standby mechanism of reading from a file, will continue to be used, instead of from shared edits, from the local edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira