Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Sun, 6 Dec 2015 01:09:11 +0000 (UTC)
From: "Arun Suresh (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12756768.1416510655000.281331.1449364151166@Atlassian.JIRA>
In-Reply-To: <JIRA.12756768.1416510655000@Atlassian.JIRA>
References: <JIRA.12756768.1416510655000@Atlassian.JIRA>
 <JIRA.12756768.1416510655637@arcas>
Subject: [jira] [Updated] (YARN-2885) Create AMRMProxy request interceptor
 for distributed scheduling decisions for queueable containers
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun Suresh updated YARN-2885:
------------------------------
    Attachment: YARN-2885-yarn-2877.002.patch

Attaching v002 of the complete patch (This also includes the API changes mentioned in the above comment)

Please Note :
# The *LocalRM* as specified in the design doc is now called the *LocalScheduler*. It is implemented as an AMRMProxy Request Interceptor
# There is a new Service running on the RM, called the *DistributedSchedulingService*. This would act as the Distributed Scheduling Coordinator. Couple of interesting things about this service:
## It implements the new *DistributedSchedulingProtocol* AS WELL as the *ApplicationMasterProtocol*. It runs an RPC server that can talk both protocols.
## It replaces the *ApplicationMasterService* when Distributed Scheduling is enabled
## It extends the *ApplicationMasterService* so that AMs can continue to talk directly to the RM (normal scheduling), but if the NM supports distributed scheduling, the AMRMProxy running on the NM (The LocalScheduler) will intercept the the register and allocate  and will talk to the RM via the *DistributedSchedulingProtocol* which essentially WRAPS over the register and allocate response with extra policy related Scheduling information and other cluster specific information (for eg. Nodes on which to schedule etc.) from the DistributedSchedulingService.
# The remainder of the changes address the suggestions made by [~leftnoteasy] and [~kkaranasos]
# I currently have just one additional test case to verify that the DistributedSchedulingService can multiplex both protocols. Will add more once we are fine with the approach
# Will add more javadocs too once we finalize the approach


> Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2885
>                 URL: https://issues.apache.org/jira/browse/YARN-2885
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Arun Suresh
>         Attachments: YARN-2885-yarn-2877.001.patch, YARN-2885-yarn-2877.002.patch, YARN-2885_api_changes.patch
>
>
> We propose to add a Local ResourceManager (LocalRM) to the NM in order to support distributed scheduling decisions. 
> Architecturally we leverage the RMProxy, introduced in YARN-2884. 
> The LocalRM makes distributed decisions for queuable containers requests. 
> Guaranteed-start requests are still handled by the central RM.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)