hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Zhang" <allenzhang...@126.com>
Subject does yarn 2.7.1 really support label expr like && or || ?
Date Thu, 17 Dec 2015 14:25:53 GMT
Hi All,


Well, my distributedshell works very well with "--shell_command ls -node_label_expression
aaa"
and I added a new label named "bbb" and tried this:
bin/hadoop jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.1.jar org.apache.hadoop.yarn.applications.distributedshell.Client
--jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.1.jar --shell_command
ls -node_label_expression "aaa || bbb"


I seems not work and it has been waiting for container for a long time. does YARN 2.7.1 really
support label expr like && or || ?






Thanks,
Allen



At 2015-12-17 22:00:29, "Allen Zhang" <allenzhang010@126.com> wrote:
>
>
>
>Response to myself:
>
>
>well, this JIRA really helped me: YARN-2502
>
>
>hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
>
>
>what surprise me is that it set "yarn.scheduler.capacity.root.accessible-node-labels"
with the label x and  set "yarn.scheduler.capacity.root.accessible-node-labels.x.capacity"
as 100
>
>
>I do think I missed that, ALSO I do think it should document somewhere in YARN doc for
other people.
>
>
>Thanks,
>Allen
>
>
>
>At 2015-12-17 16:26:19, "Allen Zhang" <allenzhang010@126.com> wrote:
>>Hi All,
>>
>>
>>I am using Hadoop 2.7.1. I have 4 vms(dev, sut-1, sut-2, sut3), but I cannot submit
my app to YARN. very appreciate for any help:)
>>
>>
>>what I did is:
>>
>>
>>Add this to yarn-site.xml file:
>><property>
>>   <name>yarn.node-labels.enabled</name>
>>   <value>true</value>
>></property>
>><property>
>>   <name>yarn.node-labels.fs-store.root-dir</name>
>>   <value>file:///tmp/</value>
>></property>
>>
>>
>>Add this to capacity-scheduler.xml file:
>><property>
>>   <name>yarn.scheduler.capacity.root.default.accessible-node-labels</name>
>>   <value>aaa</value>
>></property>
>>
>>
>>
>>
>>1. bin/yarn rmadmin -addToClusterNodeLabels aaa
>>2. bin/yarn rmadmin -replaceLabelsOnNode sut-1=aaa
>>3. bin/yarn rmadmin -refreshQueues
>>
>>
>>
>>
>>the command I was trying to submit is:
>>bin/hadoop jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.1.jar
org.apache.hadoop.yarn.applications.distributedshell.Client --jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.1.jar
--shell_command ls -node_label_expression aaa 
>>
>>
>>Thanks,
>>Allen
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message