hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2495) Allow admin specify labels in each NM (Distributed configuration)
Date Sat, 25 Oct 2014 16:09:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14184146#comment-14184146

Wangda Tan commented on YARN-2495:

if number of labels are more than the weight of heartbeat message will be more and if the
cluster nodes are more than more network IO
checking of labels from Prev state to current state for all nodes is done in RM in earlier
method each NM was taking care of it self.
Lot of read locks needs to be held @ NodesLabelsManager
Make sense, let's do that way

bq. Also as we either accept all labels or reject all labels can we have a flag whether RM
accepted the labels or not ? and modify response proto when the NodeLabelsManager Interface
changes ?
Also make sense to me, and I think a corner case is, if a NM ask for remove all labels (pass
a empty list), and how RM set reject list? So I agree to use a flag say if the last sync about
node labels is success or not

bq. RegisterNodeManagerRequestProto and NodeHeartbeatRequestProto are modified in this file
... diff shows one line after the modification which looks like i have modified RegisterNodeManagerResponseProto
Yes, you're correct, I misread the patch.

bq. So do we need to do address this scenario by adding some boolean flag for DeCentralizedConfigEnable
in RegisterNodeManagerRequestProto ?
I think we shouldn't, in such case, we should let RM follow what configured in RM side. Basically,
it's a configuration error should be avoid.
So if RM=centralized, NM=decentralized, just print error and reject such labels. 
If RM=decentralized, NM=centralized, client side will receive error message. 

Unknown macro: { createNodeStatusUpdater(context, dispatcher, nodeHealthChecker,nodeLabelsProviderService);
is it ok ?
That's fine, basically I think we should avoid modify it everywhere.


> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch, YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script suggested
by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

View raw message