hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10641) Introduce Coordination Engine
Date Fri, 06 Jun 2014 19:48:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14020284#comment-14020284
] 

Tsz Wo Nicholas Sze commented on HADOOP-10641:
----------------------------------------------

Looked at the patch.  Some questions:
- For serialization, I think we should use ProtoBuf but not java.io.Serializable.  Then, it
is easier to support compatibility.
- It seems that there is no version check between CoordinationEngine(s).  How to detect if
the engines are compatible with each other?  Also, how to support rolling upgrade?
- Do we really need Proposal and Agreement as two interfaces?  Currently, Proposal extending
Serializable is an empty interface and Agreement also extending Serializable only has one
method.
- I also don't understand why Agreement.execute(..) needs a callBackObject (learner?) parameter.
 The test does not make use of it.
- ZkCoordinationEngine also has a doExecute(..) method.  Could it be combined with Agreement.execute(..)?
 I suspect we can get rid of ZkCoordinationEngine.doExecute(..) in order to make ZkCoordinationEngine
non-abstract.  Then, applications like namenode only have to use the engine but not extend
it.
- In the CNode design doc, the Coordination Engine section describes some general concepts
and some details for the Paxos algorithm.  Could you also add some details for the ZkCoordinationEngine?
 Also, what are the advantages and disadvantages of the ZkCoordinationEngine compared with
the Paxos algorithm?

> Introduce Coordination Engine
> -----------------------------
>
>                 Key: HADOOP-10641
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10641
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HADOOP-10641.patch, HADOOP-10641.patch, HADOOP-10641.patch
>
>
> Coordination Engine (CE) is a system, which allows to agree on a sequence of events in
a distributed system. In order to be reliable CE should be distributed by itself.
> Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and
have different implementations, depending on use cases, reliability, availability, and performance
requirements.
> CE should have a common API, so that it could serve as a pluggable component in different
projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909).
> First implementation is proposed to be based on ZooKeeper.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message