hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1029) Allow embedding leader election into the RM
Date Tue, 17 Dec 2013 18:18:08 GMT

    [ https://issues.apache.org/jira/browse/YARN-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13850736#comment-13850736

Bikas Saha commented on YARN-1029:

I think we should be enable and verify ZKFC support for multiple reasons. ZKFC is kind of
standard in Hadoop HA and people are already using it. It will also validate that people can
hook up any external fencing agent (as long as it talks HAServiceProtocol).

I took a quick look at the patch
1) I dont quite agree with the ZKFCProtocol being added to the AdminService. The RM should
have only 1 interface for HA related commands to be sent to it. That interface is the HAServiceProtocol.
External ZKFC or embedded elector or the cluster admin. All of them should have the same entry
point to the RM. I dont think the RM should setup different protocols for the embedded elector
2) there are a lot of new zk related options being added here which overlap with the zkstore
options. IMO, to make this really easy to use, we should be offering zk-store and embedded
leader election as a package. If ZKStore is used then leader election can be embedded and
the store is auto-fenced. If using some other store, please use standard ZKFC which has fencing
support to fence the other RM/store. By doing this we get rid of fencing as a feature in embedded
leader election and we get rid of redundant ZK options. The elector can just reuse the ZK
options for the the store. This may also help reduce the changes needed to embed the elector.

> Allow embedding leader election into the RM
> -------------------------------------------
>                 Key: YARN-1029
>                 URL: https://issues.apache.org/jira/browse/YARN-1029
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Karthik Kambatla
>         Attachments: embedded-zkfc-approach.patch, yarn-1029-0.patch, yarn-1029-0.patch,
> It should be possible to embed common ActiveStandyElector into the RM such that ZooKeeper
based leader election and notification is in-built. In conjunction with a ZK state store,
this configuration will be a simple deployment option.

This message was sent by Atlassian JIRA

View raw message