hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2494) [YARN-796] Node label manager API and storage implementations
Date Mon, 29 Sep 2014 13:05:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151658#comment-14151658
] 

Wangda Tan commented on YARN-2494:
----------------------------------

Hi [~vinodkv] and [~cwelch],
Thanks for reply! Still working on handling your last comments, will upload patch soon.

Regarding method name of NodeLabelManager, I think following suggestion make sense to me:
bq. What I really want is to convey is that these are just system recognized nodelabels as
opposed to node-lables that are actually mapped against a node. How about addToNodeLabelsCollection(),
removeFromNodeLabelsCollection(), addLabelsToNode() and removeLabelsFromNode(). The point
about addToNodeLabelsCollection() is that it clearly conveys that there is a NodeLabelsCollection
- a set of node-labels known by the system.

And regarding
bq. Once you have the store abstraction, this will be less of a problem? Clearly NodeLabelsManager
is not something that the client needs access to?
I think it still has problem: Even if we have store abstraction, we still need some logic
to guarantee labels being added are valid (e.g. we need check if a label existed in collection,
and label existed in node when we trying to remove some labels from a node). That makes we
need put a greater chunk of logic to the store abstraction -- it isn't a simple store abstraction
if we do this.
I suggest to keep it in common to make node label major logic are live together.

Thanks,
Wangda

> [YARN-796] Node label manager API and storage implementations
> -------------------------------------------------------------
>
>                 Key: YARN-2494
>                 URL: https://issues.apache.org/jira/browse/YARN-2494
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2494.patch, YARN-2494.patch, YARN-2494.patch, YARN-2494.patch,
YARN-2494.patch, YARN-2494.patch
>
>
> This JIRA includes APIs and storage implementations of node label manager,
> NodeLabelManager is an abstract class used to manage labels of nodes in the cluster,
it has APIs to query/modify
> - Nodes according to given label
> - Labels according to given hostname
> - Add/remove labels
> - Set labels of nodes in the cluster
> - Persist/recover changes of labels/labels-on-nodes to/from storage
> And it has two implementations to store modifications
> - Memory based storage: It will not persist changes, so all labels will be lost when
RM restart
> - FileSystem based storage: It will persist/recover to/from FileSystem (like HDFS), and
all labels and labels-on-nodes will be recovered upon RM restart



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message