hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2495) Allow admin specify labels in each NM (Distributed configuration)
Date Thu, 23 Oct 2014 17:54:36 GMT

    [ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181646#comment-14181646

Wangda Tan commented on YARN-2495:

bq. But was thinking about one sceanario labels got changed and on call to NodeLabelsProvider.getLabels()
it returns the new labels but the heartbeat failed due to some reason. 
If heartbeat failed, the resource tracker in NM side cannot get NodeHeartbeatResponse. But
I'm thinking another case is, labels reported by NMs can be invalid and rejected by RM. NM
should be notified about such cases.

So I would suggest do this way:
- Keep getNodeLabels in NodeHeartbeatRequest and RegisterNodeManagerRequest.
- Add a reject node labels list in NodeHeartbeatRequest -- we may not have to handle this
list for now. But we can keep it on the interface
- Add a "lastNodeLabels" in NodeStatusUpdater, it will save last node labels list get from
NodeLabelFetcher. And in the while loop of {{startStatusUpdater}}, we will check if the new
list fetched from NodeLabelFetcher is different from our last node labels list. If different,
we will set it, if same, we will skip and set the labels to be null in next heartbeat.

And the interface of NodeLabelsProvider should be simple, just a getNodeLabels(), NodeStatusUpdater
will take care other stuffs.

bq. and for If it's distributed, AdminService should disable admin change labels on nodes
via RM admin CLI will add a jira, but was wondering how to do this ? by configuration with
new parameter?
Yes, we should add a new parameter for it, we may not need have this immediately, but we should
have one in the future. 

bq. I was earlier under the impression as MemoryRMNodeLabelsManager => is for distributed
Configuration and RMNodeLabelsManager is for Centrallized configuration. and some factory
will take care of this....
Not really, the different between them is one will persist labels to filesystem and one not.
We still have to do something for the distributed configuration.

Any thoughts? [~vinodkv]


> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script suggested
by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

View raw message