hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3964) Support NodeLabelsProvider at Resource Manager side
Date Mon, 28 Sep 2015 14:45:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14933385#comment-14933385
] 

Naganarasimha G R commented on YARN-3964:
-----------------------------------------

Hi [~dian.fu],
The scenario what is addressed as per YARN-4176 is : how to handle when {{Cluster Node label}}
which is already mapped to a node, is {{removed in RM}}. As NM will report labels to RM only
when there is any difference from previous labels sent NM, its not aware that the cluster
label is removed and there is no way NM show warning or error that label reported by it is
wrong.
But according to Delegated Node Labels solution, labels for all the nodes are updated without
verifying whether there is any change from previous state. Hence same issue doesn't arise
directly. So to minimize the impact we have default interval of 30 mins. But one doubt what
i have is, when there is large cluster like 1000 or more (practical limit what i have heard
is 50000) so what would be the load on CS on refresh of Labels on all nodes though there is
no change. 

> Support NodeLabelsProvider at Resource Manager side
> ---------------------------------------------------
>
>                 Key: YARN-3964
>                 URL: https://issues.apache.org/jira/browse/YARN-3964
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>         Attachments: YARN-3964 design doc.pdf, YARN-3964.002.patch, YARN-3964.003.patch,
YARN-3964.004.patch, YARN-3964.005.patch, YARN-3964.006.patch, YARN-3964.007.patch, YARN-3964.007.patch,
YARN-3964.008.patch, YARN-3964.009.patch, YARN-3964.010.patch, YARN-3964.011.patch, YARN-3964.012.patch,
YARN-3964.1.patch
>
>
> Currently, CLI/REST API is provided in Resource Manager to allow users to specify labels
for nodes. For labels which may change over time, users will have to start a cron job to update
the labels. This has the following limitations:
> - The cron job needs to be run in the YARN admin user.
> - This makes it a little complicate to maintain as users will have to make sure this
service/daemon is alive.
> Adding a Node Labels Provider in Resource Manager will provide user more flexibility.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message