hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10641) Introduce Coordination Engine
Date Wed, 11 Jun 2014 05:52:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027444#comment-14027444

Aaron T. Myers commented on HADOOP-10641:

bq. I just want to make sure we are on the same page here. The intent of this jira is not
to solve the general problem of distributed consensus. That is, I do not propose to build
an implementation of paxos or other coordination algorithms here. This is only to introduce
a common interface, so that real implementations such as ZooKeeper could be plugged into hadoop

Totally get that, but I think the point still remains that there's little expertise for defining
a common interface for coordination engines in general in this project, and no real reason
that the Hadoop project should necessarily be the place where that interface is defined. The
ZooKeeper project, a ZK sub-project, or an entirely new TLP makes more sense to me.

> Introduce Coordination Engine
> -----------------------------
>                 Key: HADOOP-10641
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10641
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HADOOP-10641.patch, HADOOP-10641.patch, HADOOP-10641.patch
> Coordination Engine (CE) is a system, which allows to agree on a sequence of events in
a distributed system. In order to be reliable CE should be distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and
have different implementations, depending on use cases, reliability, availability, and performance
> CE should have a common API, so that it could serve as a pluggable component in different
projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.

This message was sent by Atlassian JIRA

View raw message