hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3409) Support Node Attribute functionality
Date Mon, 24 Jul 2017 23:39:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099277#comment-16099277

Konstantinos Karanasos commented on YARN-3409:

Hi guys,

Nice to see you are resuming work on this.

I just checked the latest design document. Here are a couple of questions:
* It was not clear to me how the newly added node attributes are going to play with existing
node labels. Is the plan to share some code or will it be completely separate? I feel that
there should be some unification. Not sure I understand the two alternatives you mention ("alternate
proposal 1 & 2") compared to the solution you are proposing instead.
* Re: how applications will be specifying node attribute constraints, is the plan to use the
new constraints API we are introducing in YARN-6593?
* In the CLI API the replace and update seem a bit confusing to me. Update is essentially
adding new constraints or replacing the value of existing ones -- we could also call it set
(and even have an extra parameter that determines if we override). Replace is about removing
all existing ones and then adding new -- we could do it in two steps maybe? Also, I think
it's more intuitive to do "node1:att1=val1" instead of "node1=att1:val1".

> Support Node Attribute functionality
> ------------------------------------
>                 Key: YARN-3409
>                 URL: https://issues.apache.org/jira/browse/YARN-3409
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api, capacityscheduler, client
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: 3409-apiChanges_v2.pdf (4).pdf, Constraint-Node-Labels-Requirements-Design-doc_v1.pdf,
> Specify only one label for each node (IAW, partition a cluster) is a way to determinate
how resources of a special set of nodes could be shared by a group of entities (like teams,
departments, etc.). Partitions of a cluster has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has priority to
use the partition).
> - Percentage of capacities can apply on partition (Market team has 40% minimum capacity
and Dev team has 60% of minimum capacity of the partition).
> Attributes are orthogonal to partition, they’re describing features of node’s hardware/software
just for affinity. Some example of attributes:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >= 2.20
&& JDK.version >= 8u20 && x86_64).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message