hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subru Krishnan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2915) Enable YARN RM scale out via federation using multiple RM's
Date Tue, 02 Dec 2014 17:12:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231756#comment-14231756

Subru Krishnan commented on YARN-2915:

The proposed architecture leverages the notion of “federating” a number of standalone
YARN clusters, each with its own RM and set of NMs, in a larger federated cluster. The federated
cluster appears as a single giant cluster to the user with a single public end-point (aka
router) that confirms to the YARN application client protocol. The individual RMs will participate
as members of the federated cluster by heart beating their capabilities, enabling dynamic
self-configuration of the federated cluster. The applications running in this environment
will have access to the entire federated YARN cluster and will be able to schedule tasks on
any node of the federated cluster. We plan to extend the distributed scheduling proposed in
YARN-2877, specifically YARN-2884 to allow AMs to request containers across the federated

> Enable YARN RM scale out via federation using multiple RM's
> -----------------------------------------------------------
>                 Key: YARN-2915
>                 URL: https://issues.apache.org/jira/browse/YARN-2915
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Sriram Rao
>            Assignee: Subru Krishnan
> This is an umbrella JIRA that proposes to scale out YARN to support large clusters comprising
of tens of thousands of nodes.   That is, rather than limiting a YARN managed cluster to about
4k in size, the proposal is to enable the YARN managed cluster to be elastically scalable.

This message was sent by Atlassian JIRA

View raw message