hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10641) Introduce Coordination Engine
Date Tue, 10 Jun 2014 01:40:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026015#comment-14026015
] 

Konstantin Shvachko commented on HADOOP-10641:
----------------------------------------------

> reliable distributed coordination, which is not what Hadoop aims to do at all.

Not sure where this leaves QJM. I thought it satisfies all these requirements including the
Hadoop's aim.
Anyways, we have it, and we also do distributed coordination between active and standby NNs,
RMs.

Hadoop common is chosen for the coordination engine interface as the lowest common ancestor.
It could be used for anything Hadoop from here: HDFS, HBase, Yarn. I don't have other use
cases for it in my mind at the moment. Did not have enough experience with TLPs and Incubators,
but thought it needs like something bigger. Say, in this case,  implementations for more than
one CE. Sorry if I misunderstood you, but if you want to take it to Apache Incubator I'll
be on your side.

> Introduce Coordination Engine
> -----------------------------
>
>                 Key: HADOOP-10641
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10641
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HADOOP-10641.patch, HADOOP-10641.patch, HADOOP-10641.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of events in
a distributed system. In order to be reliable CE should be distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and
have different implementations, depending on use cases, reliability, availability, and performance
requirements.
> CE should have a common API, so that it could serve as a pluggable component in different
projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message