hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests
Date Mon, 21 Mar 2016 11:49:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204071#comment-15204071
] 

Yi Zhou commented on YARN-796:
------------------------------

Hi,
I found there can't take effect after assigning a created node label to a node in 2.6.0. You
can find that there is EMPTY in ''Node-Labels :" row by 'yarn node -status XX'. Please kindly
see below steps for details.  If there is a workaround for this issue ? Thanks advance !
step:
1) Create a Label Directory in HDFS
    sudo -u hdfs hadoop fs -mkdir -p /yarn/node-labels
    sudo -u hdfs hadoop fs -chown -R yarn:yarn /yarn
    sudo -u hdfs hadoop fs -chmod -R 700 /yarn
    sudo -u hdfs hadoop fs -ls /yarn
2) Configure YARN for Node Labels
     <property>
       <name>yarn.node-labels.enabled</name>
       <value>true</value>
    </property>
    <property>
      <name>yarn.node-labels.fs-store.root-dir</name>
      <value>hdfs://bb-node1:8020/yarn/node-labels/</value>
    </property>
3) Restart RM service
4) Add Node Labels 
    sudo -u yarn yarn rmadmin -addToClusterNodeLabels "y"
5) Assign Node Labels to Cluster Node
    sudo -u yarn yarn rmadmin -replaceLabelsOnNode "bb-node3=y"
6) Check the Node label
 sudo -u yarn yarn node -status bb-node3:8041
16/03/21 19:37:12 INFO client.RMProxy: Connecting to ResourceManager at bb-node1/192.168.30.1:8032
Node Report :
        Node-Id : bb-node3:8041
        Rack : /default
        Node-State : RUNNING
        Node-Http-Address : bb-node3:8042
        Last-Health-Update : Mon 21/Mar/16 07:36:42:522CST
        Health-Report :
        Containers : 0
        Memory-Used : 0MB
        Memory-Capacity : 65536MB
        CPU-Used : 0 vcores
        CPU-Capacity : 40 vcores
        Node-Labels :


> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>    Affects Versions: 2.4.1
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, Node-labels-Requirements-Design-doc-V1.pdf,
Node-labels-Requirements-Design-doc-V2.pdf, Non-exclusive-Node-Partition-Design.pdf, YARN-796-Diagram.pdf,
YARN-796.node-label.consolidate.1.patch, YARN-796.node-label.consolidate.10.patch, YARN-796.node-label.consolidate.11.patch,
YARN-796.node-label.consolidate.12.patch, YARN-796.node-label.consolidate.13.patch, YARN-796.node-label.consolidate.14.patch,
YARN-796.node-label.consolidate.2.patch, YARN-796.node-label.consolidate.3.patch, YARN-796.node-label.consolidate.4.patch,
YARN-796.node-label.consolidate.5.patch, YARN-796.node-label.consolidate.6.patch, YARN-796.node-label.consolidate.7.patch,
YARN-796.node-label.consolidate.8.patch, YARN-796.node-label.demo.patch.1, YARN-796.patch,
YARN-796.patch4
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels are OS,
processor architecture etc.
> We should expose these labels and allow applications to specify labels on resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message