hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests
Date Mon, 21 Mar 2016 13:33:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204184#comment-15204184
] 

Yi Zhou commented on YARN-796:
------------------------------

I configured below in capacity-scheduler.xml and try to restart RM but it always got failure
with "Illegal capacity of 1.0 for children of queue root for label=y". I think the below setting
for CS seem be OK and not very sure why it happened. Could you please help me to have a look
for this ? Thanks !

<?xml version="1.0"?>
<configuration>
 <property>
 <name>yarn.scheduler.capacity.root.queues</name>
 <value>e,m,s</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.e.capacity</name>
 <value>33</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.m.capacity</name>
 <value>34</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.s.capacity</name>
 <value>33</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.e.accessible-node-labels</name>
 <value>y</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.m.accessible-node-labels</name>
 <value>y</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.e.accessible-node-labels.y.capacity</name>
 <value>50</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.m.accessible-node-labels.y.capacity</name>
 <value>50</value>
 </property>
<property>
 <name>yarn.scheduler.capacity.root.e.default-node-label-expression</name>
 <value>y</value>
 </property>
</configuration>

Caused by: java.io.IOException: Failed to re-init queues
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:358)
	at org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:363)
	... 10 more
Caused by: java.lang.IllegalArgumentException: Illegal capacity of 1.0 for children of queue
root for label=y
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setChildQueues(ParentQueue.java:184)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:594)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:477)
	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:355)
	... 11 more

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.4.1
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, Node-labels-Requirements-Design-doc-V1.pdf,
Node-labels-Requirements-Design-doc-V2.pdf, Non-exclusive-Node-Partition-Design.pdf, YARN-796-Diagram.pdf,
YARN-796.node-label.consolidate.1.patch, YARN-796.node-label.consolidate.10.patch, YARN-796.node-label.consolidate.11.patch,
YARN-796.node-label.consolidate.12.patch, YARN-796.node-label.consolidate.13.patch, YARN-796.node-label.consolidate.14.patch,
YARN-796.node-label.consolidate.2.patch, YARN-796.node-label.consolidate.3.patch, YARN-796.node-label.consolidate.4.patch,
YARN-796.node-label.consolidate.5.patch, YARN-796.node-label.consolidate.6.patch, YARN-796.node-label.consolidate.7.patch,
YARN-796.node-label.consolidate.8.patch, YARN-796.node-label.demo.patch.1, YARN-796.patch,
YARN-796.patch4
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels are OS,
processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message