hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode
Date Mon, 02 Apr 2012 17:45:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244369#comment-13244369

Todd Lipcon commented on HDFS-3092:

bq. For the scenario you described, NN2 after it rolls JD2 and JD3, updates the JournalList
with JD2 and JD3. JD1 will no longer be used.

The situation here is at the beginning of a segment - for example the very first transaction.
So, when NN2 rolls, the starting txid of the next segment is 1. I think you need to add an
epoch number which is separate from the txid, to distinguish different "startings" of the
same segment.

bq. Fence command will include a version number that we got from the JournalList ZK node.
The number that is higher wins at the JD. The fence command with lower version # is rejected.

You'll also need to atomically write this to local storage.

I'll upload my in-progress code to HDFS-3077 so that we might be able to start sharing code
earlier rather than later.
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>                 Key: HDFS-3092
>                 URL: https://issues.apache.org/jira/browse/HDFS-3092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: MultipleSharedJournals.pdf
> Currently standby namenode relies on reading shared editlogs to stay current with the
active namenode, for namespace changes. BackupNode used streaming edits from active namenode
for doing the same. This jira is to explore using journal protocol based editlog streams for
the standby namenode. A daemon in standby will get the editlogs from the active and write
it to local edits. To begin with, the existing standby mechanism of reading from a file, will
continue to be used, instead of from shared edits, from the local edits.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message