hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6469) Coordinated replication of the namespace using ConsensusNode
Date Thu, 29 May 2014 20:51:02 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Konstantin Shvachko updated HDFS-6469:

    Attachment: CNodeDesign.pdf

In a nutshell.
Namespace updates are initiated on one of the instances of ConsensusNode (CNode), which submits
each update to the Coordination Engine as a proposal. The Coordination Engine agrees on the
order of the updates and guarantees that the updates are delivered to all CNodes as agreements
in the same order. The actual updates to the namespace are carried over as the result of executing
the corresponding agreement. This gives clients a consistent view of the namespace across
all replicas.
The replication technique provides active-active model of high availability for HDFS with
which namespace requests (read or write) can be load balanced between multiple NameNode replicas.

Coordination Engine (CE) can use different algorithms and have different implementations.
To me it would make sense to track the APIs and reference implementation(s) for CE in hadoop-common,
so that it could be used as a pluggable component in different projects, including HBase.
This jira is primarily to track the ConsensusNode development.

> Coordinated replication of the namespace using ConsensusNode
> ------------------------------------------------------------
>                 Key: HDFS-6469
>                 URL: https://issues.apache.org/jira/browse/HDFS-6469
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>         Attachments: CNodeDesign.pdf
> This is a proposal to introduce ConsensusNode - an evolution of the NameNode, which enables
replication of the namespace on multiple nodes of an HDFS cluster by means of a Coordination

This message was sent by Atlassian JIRA

View raw message