hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2800) Should print WARN log in both RM/RMAdminCLI side when MemoryRMNodeLabelsManager is enabled
Date Mon, 03 Nov 2014 23:06:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195317#comment-14195317
] 

Wangda Tan commented on YARN-2800:
----------------------------------

[~ozawa],
Moving it to RMStateStore is not a bad idea, since node label store itself can be treated
as a part of RM's state. However, the RMStateStore is hard coded can only use one storage
behind, which I'm a little concern about. I'm not quite agree with ZK can handle it well,
since we shouldn't assume this feature won't be used in a large cluster nor high frequency
updating labels.

Node labels updating is different from RMStateStore updating, client side can change all labels
of nodes (like 10k nodes) in one command, but there cannot be 10k application completed in
short period (say around seconds) at least for now. WAL based solution may be outperform in
such scenario, I think ZK is not a good back end for WAL storage.

> Should print WARN log in both RM/RMAdminCLI side when MemoryRMNodeLabelsManager is enabled
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-2800
>                 URL: https://issues.apache.org/jira/browse/YARN-2800
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: client, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch
>
>
> Even though we have documented this, but it will be better to explicitly print a message
in both RM/RMAdminCLI side to explicitly say that the node label being added will be lost
across RM restart.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message