hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian Fang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2923) Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup
Date Mon, 30 Mar 2015 17:44:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387065#comment-14387065
] 

Jian Fang commented on YARN-2923:
---------------------------------

That would require that our controller to monitor the lifecycles of the resource manager and
the individual nodes, which is what we try to avoid so that our controller will not be tightly
coupled with hadoop itself. We still like to load the labels from configuration when node
manager starts up. This also avoid information loss when RM is restarted. BTW, would the labels
in RM persist some somewhere to handle RM restart and how about the failover in HA case then?
Would the labels are reported to both RMs or there is a shared label store (or in the shared
RM state store)?

BTW, seems to me it is not a good idea to periodically check the configuration file. Why not
monitor the configuration file by some file watcher instead? Furthermore, HADOOP-7001 defines
an API to reload the configuration without restart. Would it possible to integrate HADOOP-7001
with your jira? 

> Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration
Setup 
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2923
>                 URL: https://issues.apache.org/jira/browse/YARN-2923
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2923.20141204-1.patch, YARN-2923.20141210-1.patch, YARN-2923.20150328-1.patch
>
>
> As part of Distributed Node Labels configuration we need to support Node labels to be
configured in Yarn-site.xml. And on modification of Node Labels configuration in yarn-site.xml,
NM should be able to get modified Node labels from this NodeLabelsprovider service without
NM restart



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message