hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6469) Coordinated replication of the namespace using ConsensusNode
Date Tue, 01 Jul 2014 19:30:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049227#comment-14049227
] 

Sanjay Radia commented on HDFS-6469:
------------------------------------

Wrt to double journaling
bq. If I follow your logic correctly QJM being Paxos-based uses a journal by itself, so we
are not increasing journaling here. When you look at the bigger picture we see more journals
around. HBase uses WAL along with NN edits, which by itself persisted in ext4 a journaling
file system.

Konstatine, Todd's point is not that there are multiple journal in the system but that every
update operation of NN, will result in an entry in two journals: HDFS's edit log and the journal
used  by the consensusNode  paxos protocol.  Your example of HBases log and NN log is not
a good comparison: every write to the HBase WAL does NOT result in a HDFS editlog entry -
an entry is made in the HDFS editlog ONLY when the WAL crosses a block boundary. 

> Coordinated replication of the namespace using ConsensusNode
> ------------------------------------------------------------
>
>                 Key: HDFS-6469
>                 URL: https://issues.apache.org/jira/browse/HDFS-6469
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>         Attachments: CNodeDesign.pdf
>
>
> This is a proposal to introduce ConsensusNode - an evolution of the NameNode, which enables
replication of the namespace on multiple nodes of an HDFS cluster by means of a Coordination
Engine.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message