hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Flavio Junqueira (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode
Date Wed, 04 Apr 2012 15:49:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246385#comment-13246385
] 

Flavio Junqueira commented on HDFS-3092:
----------------------------------------

bq. Is there a way to turn off the striping even if Quorom size (Q) is less than Ensemble
size (E)?
We like the idea that each Journal file contains ALL entries.
Our default config: Q is 2 and set of JDs is 3 (roughly equivalent to E).

Right now the write set is the same as the ack set, since we haven't had the use case you
suggest so far. We have anticipated the need of different ways of scheduling reads and writes,
and it is fairly simple to make it cover your use case. Ivan had actually suggested this separation
between ack sets and write sets previously to deal with slow disks. 

I have tried to reflect this discussion in BOOKKEEPER-208 if you're interested.  
                
> Enable journal protocol based editlog streaming for standby namenode
> --------------------------------------------------------------------
>
>                 Key: HDFS-3092
>                 URL: https://issues.apache.org/jira/browse/HDFS-3092
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 0.24.0, 0.23.3
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: MultipleSharedJournals.pdf, MultipleSharedJournals.pdf
>
>
> Currently standby namenode relies on reading shared editlogs to stay current with the
active namenode, for namespace changes. BackupNode used streaming edits from active namenode
for doing the same. This jira is to explore using journal protocol based editlog streams for
the standby namenode. A daemon in standby will get the editlogs from the active and write
it to local edits. To begin with, the existing standby mechanism of reading from a file, will
continue to be used, instead of from shared edits, from the local edits.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message