hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3632) Ordering policy should be allowed to reorder an application when demand changes
Date Tue, 19 May 2015 17:43:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550852#comment-14550852
] 

Wangda Tan commented on YARN-3632:
----------------------------------

I think we can add a set to queue to track apps (schedulableEntity) needs to be changed, we
don't need to remove/insert it everytime, we only need to do that once when doing assignContainers
next time. Pesudo code may look like:
{code}
if (schedulableEntity.allocate-container/release-container/update-demand) then:
   orderingPolicy.markNeedUpdate(schedulableEntity)
{code}

And

{code}
orderingPolicy#getAllocateIterator:
   for (schedulableEntity : needToUpdateEntities):
        remove-and-insert(schedulableEntity)
{code}

This can avoid excessive modifications to TreeSet in OrderingPolicy. Thoughts? [~cwelch].

> Ordering policy should be allowed to reorder an application when demand changes
> -------------------------------------------------------------------------------
>
>                 Key: YARN-3632
>                 URL: https://issues.apache.org/jira/browse/YARN-3632
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>         Attachments: YARN-3632.0.patch, YARN-3632.1.patch, YARN-3632.3.patch, YARN-3632.4.patch,
YARN-3632.5.patch
>
>
> At present, ordering policies have the option to have an application re-ordered (for
allocation and preemption) when it is allocated to or a container is recovered from the application.
 Some ordering policies may also need to reorder when demand changes if that is part of the
ordering comparison, this needs to be made available (and used by the fairorderingpolicy when
sizebasedweight is true)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message