hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3388) Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit
Date Tue, 14 Jun 2016 15:19:01 GMT

     [ https://issues.apache.org/jira/browse/YARN-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nathan Roberts updated YARN-3388:
---------------------------------
    Attachment: YARN-3388-v3.patch

[~leftnoteasy], [~eepayne]. Ok, "soon" was extremely relative;) Sorry about that. 

I think I addressed Wangda's comments but I need label partition experts to take a look.

Any ideas why people don't hit this more often? We find it's very easy to get stuck at queueCapacity
even though userLimitFactor and maxCapacity say the system should allocate further. Do you
think people aren't using DRF and are mostly just using memory as the resource?

> Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when
computing user-limit
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3388
>                 URL: https://issues.apache.org/jira/browse/YARN-3388
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler
>    Affects Versions: 2.6.0
>            Reporter: Nathan Roberts
>            Assignee: Nathan Roberts
>         Attachments: YARN-3388-v0.patch, YARN-3388-v1.patch, YARN-3388-v2.patch, YARN-3388-v3.patch
>
>
> When there are multiple active users in a queue, it should be possible for those users
to make use of capacity up-to max_capacity (or close). The resources should be fairly distributed
among the active users in the queue. This works pretty well when there is a single resource
being scheduled.   However, when there are multiple resources the situation gets more complex
and the current algorithm tends to get stuck at Capacity. 
> Example illustrated in subsequent comment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message