hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-3409) Add constraint node labels
Date Thu, 08 Dec 2016 19:14:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733152#comment-15733152
] 

Wangda Tan edited comment on YARN-3409 at 12/8/16 7:14 PM:
-----------------------------------------------------------

And,

bq. do you mean having an extra affinity/anti-affinity constraint expression or use the same
constraint expression? Probably we will need a separate one.
I would prefer to use the same one, this is why I suggest to name the field to "placementStrategy"
instead of "constraintExpression". To me YARN-4902 is the goal, we should align new changes
to RequestRequest to the new proposal.
So example of how to specify anti-affinity and constraint may look like:
{code}
# Run on node with JDK5 or JDK6, and anti-affinity to itself's container
placement-strategy = (JDK5 || JDK6) && anti-affinity-to <my-application>

# Run on node with CENTOS6 and SSD, and affinity-to application-2 
placement-strategy = CENTOS6 && SSD && affinity-to application_3333_0002
{code}

To make scheduler can simpler handle it, we can limit number of affinity and anti-affinity
constraint within one placement-strategy to <= 1. (You cannot specify {{(JDK5 &&
affinity-to app_1) || (JDK6 && anti-affinity-to app_2)}}.

Please share your thoughts on this.


was (Author: leftnoteasy):
And,

bq. do you mean having an extra affinity/anti-affinity constraint expression or use the same
constraint expression? Probably we will need a separate one.
I would prefer to use the same one, this is why I suggest to name the field to "placementStrategy"
instead of "constraintExpression". To me YARN-4902 is the goal, we should align new changes
to RequestRequest to the new proposal.
So example of how to specify anti-affinity and constraint may look like:
{code}
# Run on node with JDK5 or JDK6, and anti-affinity to itself's container
placement-strategy = (JDK5 || JDK6) && anti-affinity-to <my-application>

# Run on node with CENTOS6 and SSD, and affinity-to application-2 
placement-strategy = CENTOS6 && SSD && affinity-to application_3333_0002
{code} 

Please share your thoughts on this.

> Add constraint node labels
> --------------------------
>
>                 Key: YARN-3409
>                 URL: https://issues.apache.org/jira/browse/YARN-3409
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, capacityscheduler, client
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf, YARN-3409.WIP.001.patch
>
>
> Specify only one label for each node (IAW, partition a cluster) is a way to determinate
how resources of a special set of nodes could be shared by a group of entities (like teams,
departments, etc.). Partitions of a cluster has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has priority to
use the partition).
> - Percentage of capacities can apply on partition (Market team has 40% minimum capacity
and Dev team has 60% of minimum capacity of the partition).
> Constraints are orthogonal to partition, they’re describing attributes of node’s
hardware/software just for affinity. Some example of constraints:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >= 2.20
&& JDK.version >= 8u20 && x86_64).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message