hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3409) Add constraint node labels
Date Mon, 05 Dec 2016 00:21:59 GMT

     [ https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Naganarasimha G R updated YARN-3409:
    Attachment: YARN-3409.WIP.001.patch

I am attaching the first WIP patch. Here i have briefly captured how the constraints evaluation
happens involving system default constraint types and for the custom constraint type. As described
in the document i have supported "string", "double", "long" & "boolean" as system default
constraint type. {{"ConstraintExpressionParser"}} is responsible for parsing the constraint
expression. It parses the expression based on the format {{" compareExpression <expressionOperator>
compareExpression "}}. expressionOperators supported are "||" / "or" and "&&" / "and".
And Compare expression is of the format.  {{"constraintName <compareOperator> constraintValueExpression"}}.
compareOperators which are supported are 
  LESS_THAN, // '<' or 'lt'
  LESS_OR_EQUAL, // '<=' or 'le'
  EQUAL, // '==' or 'eq'
  NOT_EQUAL, // '!=' or 'ne' or 'ene' (for exists but not equal)
  GREATER_OR_EQUAL, // '>=' or 'ge'
  GREATER_THAN, // '>' or 'gt'
  IN, // 'in'
  NOT_IN, // 'not_in'
  EXISTS, // no operand only if the name is specified
  NOT_EXISTS, // '! <constrainName>'

For more examples please refer to {{TestConstraintExpressionParser}} which provides some examples
for expressions and evaluations and {{ConstraintExpressionParser}} for some documentation.
Will try to update in WIP with the CLI commands with trial modification in the RequestResource
interface for understanding the impacts of the Constraints.

I thought i had captured for ResourceRequest API with all options (with pro's and con's )
and hence i concluded "option b" is better. May be if i missed anything there or is not clear,
i will be glad to restate in the next version of the design doc.
For the CLI and REST, IIUC we do not support REST for rm admin functionalities and will be
there for only query, and i share the same opinion as [~varun_saxena] that they are different.
Will upload the next CLI patch so that its more clear why we thought of segregating it.

> Add constraint node labels
> --------------------------
>                 Key: YARN-3409
>                 URL: https://issues.apache.org/jira/browse/YARN-3409
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, capacityscheduler, client
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf, YARN-3409.WIP.001.patch
> Specify only one label for each node (IAW, partition a cluster) is a way to determinate
how resources of a special set of nodes could be shared by a group of entities (like teams,
departments, etc.). Partitions of a cluster has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has priority to
use the partition).
> - Percentage of capacities can apply on partition (Market team has 40% minimum capacity
and Dev team has 60% of minimum capacity of the partition).
> Constraints are orthogonal to partition, they’re describing attributes of node’s
hardware/software just for affinity. Some example of constraints:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >= 2.20
&& JDK.version >= 8u20 && x86_64).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message