hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2885) Create AMRMProxy request interceptor for distributed scheduling decisions for queueable containers
Date Thu, 03 Dec 2015 07:00:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037404#comment-15037404
] 

Konstantinos Karanasos commented on YARN-2885:
----------------------------------------------

Thank you for the patch, [~asuresh].
Adding some more comments to this first version:
# Given that the list of nodes to be used for distributed scheduling ("top-k nodes") is ordered,
we need to send the whole list at each AllocateResponse (it will become complicated to do
so by sending just the delta of the list in the form of new/removed nodes).
# Given the above point, we will not need to have a node list in the RegisterApplicationMasterResponse.
# I suggest to remove the two parameters for setting limits to the number of QUEUEABLE containers
from this JIRA, since YARN-2889 targets this functionality.
# I propose to remove the support for locality from this first version of the JIRA. Getting
it right requires more work (given that each LocalRM only sees a subset of the cluster's nodes),
and should probably be the objective of a separate sub-JIRA.
# When creating the Interceptor chain in the AMRMProxyService, make sure the DistSchedulerRequestInterceptor
is always placed in the beginning of the chain.
# We could make DistSchedulerParameters a subclass to the DistSchedulerRequestInterceptor
rather than a separate class.

> Create AMRMProxy request interceptor for distributed scheduling decisions for queueable
containers
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2885
>                 URL: https://issues.apache.org/jira/browse/YARN-2885
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Arun Suresh
>         Attachments: YARN-2885-yarn-2877.001.patch
>
>
> We propose to add a Local ResourceManager (LocalRM) to the NM in order to support distributed
scheduling decisions. 
> Architecturally we leverage the RMProxy, introduced in YARN-2884. 
> The LocalRM makes distributed decisions for queuable containers requests. 
> Guaranteed-start requests are still handled by the central RM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message