hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5342) Improve non-exclusive node partition resource allocation in Capacity Scheduler
Date Thu, 14 Jul 2016 00:50:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376095#comment-15376095
] 

Wangda Tan commented on YARN-5342:
----------------------------------

[~Naganarasimha], that's a good point, actually I have thought about this while working on
the patch.
The only purpose of doing this is for simple, we could have some better logic like gradually
decrease the counter depends on ratio of #nodes in default partition and #nodes in specific
partitions, but they could be complex and potentially can be a regression since we don't know
what happened. Please share your thoughts. 

Thanks,

> Improve non-exclusive node partition resource allocation in Capacity Scheduler
> ------------------------------------------------------------------------------
>
>                 Key: YARN-5342
>                 URL: https://issues.apache.org/jira/browse/YARN-5342
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-5342.1.patch
>
>
> In the previous implementation, one non-exclusive container allocation is possible when
the missed-opportunity >= #cluster-nodes. And missed-opportunity will be reset when container
allocated to any node.
> This will slow down the frequency of container allocation on non-exclusive node partition:
*When a non-exclusive partition=x has idle resource, we can only allocate one container for
this app in every X=nodemanagers.heartbeat-interval secs for the whole cluster.*
> In this JIRA, I propose a fix to reset missed-opporunity only if we have >0 pending
resource for the non-exclusive partition OR we get allocation from the default partition.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message