hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10641) Introduce Coordination Engine
Date Mon, 09 Jun 2014 17:27:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025388#comment-14025388
] 

Aaron T. Myers commented on HADOOP-10641:
-----------------------------------------

bq. Aaron, a separate project is a great idea. 

Cool, glad we agree. Shall we resolve this JIRA then and take this proposal to the Apache
Incubator? A standalone separate project seems like a much more reasonable place to put this
work to me.

bq. ZK is probably not the best home for the interface as ZKCoordinationEngine is only one
its implementations. People talked to me about possible implementations using Raft, Bookeeper,
Paxos, and two other engines. I think the variety makes sense as there could be different
performance, network, reliability etc. requirements. Having said that, I am flexible if ZK
community wants to adopt it.

I was only suggesting ZK since that project's focus is about reliable distributed coordination,
which is not what Hadoop aims to do at all. If for some reason you didn't want to try to make
this a TLP (which, again, seems more reasonable to me) then trying to contribute it to that
project makes a lot more sense to me.

> Introduce Coordination Engine
> -----------------------------
>
>                 Key: HADOOP-10641
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10641
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HADOOP-10641.patch, HADOOP-10641.patch, HADOOP-10641.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of events in
a distributed system. In order to be reliable CE should be distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and
have different implementations, depending on use cases, reliability, availability, and performance
requirements.
> CE should have a common API, so that it could serve as a pluggable component in different
projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message