hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4425) Pluggable sharing policy for Partition Node Label resources
Date Fri, 11 Dec 2015 18:56:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053213#comment-15053213
] 

Wangda Tan commented on YARN-4425:
----------------------------------

Thanks [~Naganarasimha]/[~xinxianyin],

I took a quick look at attached doc, this is a very interesting proposal.

I think the doc listed pain points of existing non-exclusive node label implementation. Now
usage of non-exclusive node label is limited (only supports non-labeled request allocated
on non-exclusive partitions with long delays). It's important to support labeled-request also
and change the behavior of deciding when to share resources.

I felt proposed solution should be doable, maybe it will be more helpful to understand what's
the scope after you finish doing a POC.

Only one comment on the proposal is:
I can understand why we need two separate policies, maybe we can consider to merge two policies
to one: currently exclusivity of node label is just a hint, sharing policy could decide if
and how much resources could be shared from each partition.

> Pluggable sharing policy for Partition Node Label resources
> -----------------------------------------------------------
>
>                 Key: YARN-4425
>                 URL: https://issues.apache.org/jira/browse/YARN-4425
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler, resourcemanager
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: ResourceSharingPolicyForNodeLabelsPartitions-V1.pdf
>
>
> As part of support for sharing NonExclusive Node Label partitions in YARN-3214, NonExclusive
partitions are shared only to Default Partitions and also have fixed rule when apps in default
partitions makes use of resources of any NonExclusive partitions.
> There are many scenarios where in we require pluggable policy like MutliTenant, Hierarchical
etc.. where in each partition can determine when they want to share the resources to other
paritions and when other partitions wants to use resources from others
> More details in the attached document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message