hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1029) Allow embedding leader election into the RM
Date Wed, 11 Dec 2013 23:53:08 GMT

    [ https://issues.apache.org/jira/browse/YARN-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13845858#comment-13845858

Vinod Kumar Vavilapalli commented on YARN-1029:

I think so too. Typical installations will need YARN-1177 more than this option? So sequence
that first?

Re this patch, seems like embedding ZKFC is beneficial.

bq. However, automatic failover fails to take over after an explicit manual failover. To address
this RMActiveStandbyElector should implement ZKFCProtocol and RMHAServiceTarget#getZKFCProxy
should return a proxy to this.
bq. In addition to ActiveStandbyElector, ZKFC has other overheads - health monitoring, fencing
etc. which might not be required in a simple embedded option.I see that 
ZKFC = health-monitoring + leader election + graceful failover (ZKFCProtocol). Seems like
for the embedded case, we want to use leader-election + fencing. To that end, may be we should
refactor ZKFC itself for reuse?

bq. ZKFC communicates to the RM through RPC; when embedded, both are in the same process.
We've done similar local RPC short-circuits for token renewal. That should fix it?

bq. ZKFC#formatZK() needs to be exposed through rmadmin, which complicates it further.
If I understand it correctly, it can be implemented as a standalone command instead of a RMAdmin
call. Right?

> Allow embedding leader election into the RM
> -------------------------------------------
>                 Key: YARN-1029
>                 URL: https://issues.apache.org/jira/browse/YARN-1029
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Karthik Kambatla
>         Attachments: yarn-1029-approach.patch
> It should be possible to embed common ActiveStandyElector into the RM such that ZooKeeper
based leader election and notification is in-built. In conjunction with a ZK state store,
this configuration will be a simple deployment option.

This message was sent by Atlassian JIRA

View raw message