hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers
Date Sun, 06 Dec 2015 01:09:11 GMT

     [ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arun Suresh updated YARN-2885:
    Attachment: YARN-2885-yarn-2877.002.patch

Attaching v002 of the complete patch (This also includes the API changes mentioned in the
above comment)

Please Note :
# The *LocalRM* as specified in the design doc is now called the *LocalScheduler*. It is implemented
as an AMRMProxy Request Interceptor
# There is a new Service running on the RM, called the *DistributedSchedulingService*. This
would act as the Distributed Scheduling Coordinator. Couple of interesting things about this
## It implements the new *DistributedSchedulingProtocol* AS WELL as the *ApplicationMasterProtocol*.
It runs an RPC server that can talk both protocols.
## It replaces the *ApplicationMasterService* when Distributed Scheduling is enabled
## It extends the *ApplicationMasterService* so that AMs can continue to talk directly to
the RM (normal scheduling), but if the NM supports distributed scheduling, the AMRMProxy running
on the NM (The LocalScheduler) will intercept the the register and allocate  and will talk
to the RM via the *DistributedSchedulingProtocol* which essentially WRAPS over the register
and allocate response with extra policy related Scheduling information and other cluster specific
information (for eg. Nodes on which to schedule etc.) from the DistributedSchedulingService.
# The remainder of the changes address the suggestions made by [~leftnoteasy] and [~kkaranasos]
# I currently have just one additional test case to verify that the DistributedSchedulingService
can multiplex both protocols. Will add more once we are fine with the approach
# Will add more javadocs too once we finalize the approach

> Create AMRMProxy request interceptor for distributed scheduling decisions for queueable
> --------------------------------------------------------------------------------------------------
>                 Key: YARN-2885
>                 URL: https://issues.apache.org/jira/browse/YARN-2885
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Arun Suresh
>         Attachments: YARN-2885-yarn-2877.001.patch, YARN-2885-yarn-2877.002.patch, YARN-2885_api_changes.patch
> We propose to add a Local ResourceManager (LocalRM) to the NM in order to support distributed
scheduling decisions. 
> Architecturally we leverage the RMProxy, introduced in YARN-2884. 
> The LocalRM makes distributed decisions for queuable containers requests. 
> Guaranteed-start requests are still handled by the central RM.

This message was sent by Atlassian JIRA

View raw message