hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2495) Allow admin specify labels from each NM (Distributed configuration)
Date Tue, 10 Mar 2015 18:40:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355436#comment-14355436
] 

Wangda Tan commented on YARN-2495:
----------------------------------

Hi Naga,
--
bq. Well modifications side is clear to me but is it good to allow the configurations being
different from NM and RM ? Infact i wanted to discuss regarding whether to send shutdown during
register if NM is configured differently from RM, but waited for the base changes to go in
before discussing new stuff.

It is not make the configuration different, my thinking of this is, NM doesn't need understand
what is "distribtued-configuration", admin should know it. When the node-label is "distributed-configuration",
NM should go ahead and properly configure script provider, etc.
So we aren't trying to create different, we just eliminate one option in NM side.

--
bq.  I feel better to Log this as "Error" as we are sending the labels only in case of any
change and there has to be some way to identify if labels for a given NM and also currently
we are sending out shutdown signal too.

What I meant are the two line,
{code}
498	      LOG.info("Node Labels {" + StringUtils.join(",", nodeLabels)
499	          + "} from Node " + nodeId + " were Accepted from RM");
{code}
I guess you may misread my comment

--
For the field to indicate if node labels are set in NodeHeartbeatRequest/NodeRegistrationRequest,
there're two proposals: 
- setAreNodeLabelsSetInReq
- setAreNodeLabelsUpdated
Which one you prefer, Vinod/Craig? I vote the latter one :)

--
bq. We should not even accept a node's registration when it reports invalid labels
IIUC, now the patch already reject node when it reports invalid labels.

> Allow admin specify labels from each NM (Distributed configuration)
> -------------------------------------------------------------------
>
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch, YARN-2495.20141030-1.patch,
YARN-2495.20141031-1.patch, YARN-2495.20141119-1.patch, YARN-2495.20141126-1.patch, YARN-2495.20141204-1.patch,
YARN-2495.20141208-1.patch, YARN-2495.20150305-1.patch, YARN-2495.20150309-1.patch, YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml (YARN-2923) or using script
suggested by [~aw] (YARN-2729) )
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message