hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6964) Fair scheduler misuses Resources operations
Date Tue, 08 Aug 2017 01:28:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117694#comment-16117694

Miklos Szegedi commented on YARN-6964:

Thanks for the patch [~templedf].
566	    Resource desiredShare =
567	        Resources.componentwiseMin(getMinShare(), getDemand());
I am wondering, if this is the right thing to do. Let's say I have a container that can run
6 threads parallel and it needs a 1GB buffer per thread. Now I request 6 vcores and 6 GB but
my min share is just 3 GB and 6 vcores. The cluster has 1000GB and 1000 vcores. I will get
a container with 6 vcores, 3 of which I cannot use. The "dominant" in terms of min share is
3GB, so in case of DRF, I should get 3GB and 3 vcores.

> Fair scheduler misuses Resources operations
> -------------------------------------------
>                 Key: YARN-6964
>                 URL: https://issues.apache.org/jira/browse/YARN-6964
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 3.0.0-alpha4
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>         Attachments: YARN-6964.001.patch
> There are several places where YARN uses the {{Resources}} class to do comparisons of
{{Resource}} instances incorrectly.  This patch corrects those mistakes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message