hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode
Date Wed, 09 May 2012 04:57:53 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271098#comment-13271098
] 

Eli Collins commented on HDFS-3092:
-----------------------------------

Notes from the meeting:
* Discussed the two approaches for the client side (HDFS-3077 and HDFS-3092), the server sides
are similar modulo small differences in recovery. Discussed the tradeoffs in terms of going
down to n-1 servers/requiring a quorum vs latency sensitivity, in the context of typical cluster
configurations. Think it's possible to wed the client side of 3077 (quorum journal) to the
server side of 3092 (journal daemon). Will pursue further on jira.
* Discussed journal-based fencing. Current NN fencers are not needed when the journal handles
fencing.
* Discussed IP based failover and stonith, primary motivation for the current approach is
that other master services that are not yet HA often run on the same machines, stonith doesn't
work here.
* Discussed making the state machine for auto failover more explicit
* Discussed separate vs embedded FCs, separate works well for now, though currently means
we'll need a FC per service that gets failover (vs embedding a FC in each service that will
need failover)
                
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>
>                 Key: HDFS-3092
>                 URL: https://issues.apache.org/jira/browse/HDFS-3092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: ComparisonofApproachesforHAJournals.pdf, JNStates.png, MultipleSharedJournals.pdf,
MultipleSharedJournals.pdf, MultipleSharedJournals.pdf, Removingfilerdependency.pdf
>
>
> Currently standby namenode relies on reading shared editlogs to stay current with the
active namenode, for namespace changes. BackupNode used streaming edits from active namenode
for doing the same. This jira is to explore using journal protocol based editlog streams for
the standby namenode. A daemon in standby will get the editlogs from the active and write
it to local edits. To begin with, the existing standby mechanism of reading from a file, will
continue to be used, instead of from shared edits, from the local edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message