hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2800) Should print WARN log in both RM/RMAdminCLI side when MemoryRMNodeLabelsManager is enabled
Date Tue, 04 Nov 2014 00:00:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195407#comment-14195407
] 

Tsuyoshi OZAWA commented on YARN-2800:
--------------------------------------

[~leftnoteasy], If we assume the labels as a configuration which can be highly updated, ZK
is not good option as you mentioned. In this case, I think NodeLabelsManager, whose backend
can be leveldb or rockdb, should be loosely coupling with RM like TimelineServer for stabilization
of RM. One option is making NodeLabelsManager NodeLabelsServer.  It means RM should work correctly
even if NodeLabelsManager is temporary unavailable. And update operation should only affect
NodeLabelsManager(it doesn't affect RM). For example, RM pulls the label information from
NodeLabelsServer periodically. RM treats the lable information as a hint and does schedule
based on label information. Even without the information, RM should schedule apps. I think
this weak consistency approach is suitable for large-scale updating.

> Should print WARN log in both RM/RMAdminCLI side when MemoryRMNodeLabelsManager is enabled
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-2800
>                 URL: https://issues.apache.org/jira/browse/YARN-2800
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: client, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch
>
>
> Even though we have documented this, but it will be better to explicitly print a message
in both RM/RMAdminCLI side to explicitly say that the node label being added will be lost
across RM restart.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message