hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4898) Avoid sending NODE_LABELS_UPDATE event to scheduler when node label is not configured
Date Tue, 05 Apr 2016 16:47:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226620#comment-15226620
] 

Naganarasimha G R commented on YARN-4898:
-----------------------------------------

Thanks [~rohithsharma] for raising this issue, 
Seems like we do not require most of the functionalities done by  {{RMNodelableManager}} when
node labels are not enabled. So i will try to evaluate the impacts if the {{RMNodelableManager}}
is set to null or have a overridden implementation which can avoid the redundant functionalities
when the labels are disabled.
i would like to take this up if you have not already started.

> Avoid sending NODE_LABELS_UPDATE event to scheduler when node label is not configured
> -------------------------------------------------------------------------------------
>
>                 Key: YARN-4898
>                 URL: https://issues.apache.org/jira/browse/YARN-4898
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>            Reporter: Rohith Sharma K S
>
> It is observed that when ever new NodeManager is registered or expired with ResourceManager,
a new additional event NODE_LABELS_UPDATE has been triggered even though *Node Label is not
enabled*. This makes dispatcher to process one event without any real use. 
> logs : where all 100 of node-labels-update events processed by dispatcher. 
> {noformat}
> 2016-03-30 15:42:24,461 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175))
- Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType:
NODE_LABELS_UPDATE
> 2016-03-30 15:42:24,461 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175))
- Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType:
NODE_LABELS_UPDATE
> 2016-03-30 15:42:24,462 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175))
- Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType:
NODE_LABELS_UPDATE
> 2016-03-30 15:42:24,462 DEBUG [AsyncDispatcher event handler] event.AsyncDispatcher (AsyncDispatcher.java:dispatch(175))
- Dispatching the event org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeLabelsUpdateSchedulerEvent.EventType:
NODE_LABELS_UPDATE
> {noformat}
> Point of concern is In large cluster, it is seen that registering thousands of nodes
altogether to running cluster might/would cause other events processing to get delayed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message