hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kishore Chaliparambil (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2884) Proxying all AM-RM communications
Date Thu, 23 Jul 2015 01:13:06 GMT

     [ https://issues.apache.org/jira/browse/YARN-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Kishore Chaliparambil updated YARN-2884:
    Attachment: YARN-2884-V1.patch

Added a new service in NodeManager called AMRMProxyService. This is by default disabled and
can be enabled using "yarn.nodemanager.amrmproxy.enable". This service can be used for proxying
calls from the Application Master to the Resource Manager. 

This feature is needed for:
1) Federation - to transparently scale the application across multiple sub-clusters (YARN-2915)
2) Mercury – to queue containers directly in NMs for distributed scheduling (YARN-2877)
3) Security/Throttling etc. – prevent DDoS attacks on RM

> Proxying all AM-RM communications
> ---------------------------------
>                 Key: YARN-2884
>                 URL: https://issues.apache.org/jira/browse/YARN-2884
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Carlo Curino
>            Assignee: Kishore Chaliparambil
>         Attachments: YARN-2884-V1.patch
> We introduce the notion of an RMProxy, running on each node (or once per rack). Upon
start the AM is forced (via tokens and configuration) to direct all its requests to a new
services running on the NM that provide a proxy to the central RM. 
> This give us a place to:
> 1) perform distributed scheduling decisions
> 2) throttling mis-behaving AMs
> 3) mask the access to a federation of RMs

This message was sent by Atlassian JIRA

View raw message