Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 23 Oct 2014 17:54:36 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12738746.1409734418000.324372.1414086876177@Atlassian.JIRA>
In-Reply-To: <JIRA.12738746.1409734418000@Atlassian.JIRA>
References: <JIRA.12738746.1409734418000@Atlassian.JIRA>
 <JIRA.12738746.1409734418612@arcas>
Subject: [jira] [Commented] (YARN-2495) Allow admin specify labels in each
 NM (Distributed configuration)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181646#comment-14181646 ] 

Wangda Tan commented on YARN-2495:
----------------------------------

1) 
bq. But was thinking about one sceanario labels got changed and on call to NodeLabelsProvider.getLabels() it returns the new labels but the heartbeat failed due to some reason. 
If heartbeat failed, the resource tracker in NM side cannot get NodeHeartbeatResponse. But I'm thinking another case is, labels reported by NMs can be invalid and rejected by RM. NM should be notified about such cases.

So I would suggest do this way:
- Keep getNodeLabels in NodeHeartbeatRequest and RegisterNodeManagerRequest.
- Add a reject node labels list in NodeHeartbeatRequest -- we may not have to handle this list for now. But we can keep it on the interface
- Add a "lastNodeLabels" in NodeStatusUpdater, it will save last node labels list get from NodeLabelFetcher. And in the while loop of {{startStatusUpdater}}, we will check if the new list fetched from NodeLabelFetcher is different from our last node labels list. If different, we will set it, if same, we will skip and set the labels to be null in next heartbeat.

And the interface of NodeLabelsProvider should be simple, just a getNodeLabels(), NodeStatusUpdater will take care other stuffs.

2)
bq. and for If it's distributed, AdminService should disable admin change labels on nodes via RM admin CLI will add a jira, but was wondering how to do this ? by configuration with new parameter?
Yes, we should add a new parameter for it, we may not need have this immediately, but we should have one in the future. 

bq. I was earlier under the impression as MemoryRMNodeLabelsManager => is for distributed Configuration and RMNodeLabelsManager is for Centrallized configuration. and some factory will take care of this....
Not really, the different between them is one will persist labels to filesystem and one not. We still have to do something for the distributed configuration.

Any thoughts? [~vinodkv]

Thanks,
Wangda

> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)