Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Mon, 21 Jul 2014 03:41:40 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12652117.1370907497973.6004.1405914100476@arcas>
In-Reply-To: <JIRA.12652117.1370907497973@arcas>
References: <JIRA.12652117.1370907497973@arcas>
Subject: [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and
 resource-requests
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068163#comment-14068163 ] 

Wangda Tan commented on YARN-796:
---------------------------------

Hi [~sunilg],
bq. 2. Regarding reservations, how about introducing node-label reservations. Ideas is like, if an application is lacking resource on a node, it can reserve on that node as well as to node-label. So when a suitable node update comes from another node in same node-label, can try allocating container in new node by unreserving from old node.
I think this makes sense, we'd better support this. I will check our current resource reservation/unreservation logic how to support it, will keep you posted.

bq. 3. My approach was more like have a centralized configuration, but later after some time, if want to add a new node to cluster, then it can start with a hardcoded label in its yarn-site. In your approach, we need to use REStful API or admin command to bring this node under one label. May be while start up itself this node can be set under a label. your thoughts?
I think a problem of mixed centralized/distributed configuration I can see is, it will be hard to manage them after RM/NM restart -- should we use labels specified in NM config or our centralized config? I also replied Jian Fang previously about this: https://issues.apache.org/jira/browse/YARN-796?focusedCommentId=14063316&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14063316.
Maybe a workaround is we can define the centralized config all always overwrite distributed config. E.g. user defined "GPU" in NM config, and admin use RESTful added "FPGA", RM will serialize both "GPU", "FPGA" into a centralized storage system. And after RM restart or NM restart, RM will ignore NM config if anything defined in RM. But I still think it's better to avoid use both of them together.

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, Node-labels-Requirements-Design-doc-V1.pdf, YARN-796.patch
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels are OS, processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.


--
This message was sent by Atlassian JIRA
(v6.2#6252)