hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode
Date Mon, 23 Apr 2012 05:39:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259394#comment-13259394
] 

dhruba borthakur commented on HDFS-3092:
----------------------------------------

I am trying to digest the meat of this approach, but one question that I do not have an answer
for: is it possible for the journal daemon to write data to disks and nodes that do not share
load from other non-journal writers? I feel this requirement will be critical to ensure low
variance of write latencies for the journal. My experience is that a 5% increase in the latency
of writes to the transaction log causes a 20% degradation of namenode throughput in a large
cluster.
                
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>
>                 Key: HDFS-3092
>                 URL: https://issues.apache.org/jira/browse/HDFS-3092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: ComparisonofApproachesforHAJournals.pdf, MultipleSharedJournals.pdf,
MultipleSharedJournals.pdf, MultipleSharedJournals.pdf
>
>
> Currently standby namenode relies on reading shared editlogs to stay current with the
active namenode, for namespace changes. BackupNode used streaming edits from active namenode
for doing the same. This jira is to explore using journal protocol based editlog streams for
the standby namenode. A daemon in standby will get the editlogs from the active and write
it to local edits. To begin with, the existing standby mechanism of reading from a file, will
continue to be used, instead of from shared edits, from the local edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message