hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6964) Fair scheduler misuses Resources operations
Date Tue, 08 Aug 2017 16:34:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118587#comment-16118587

Daniel Templeton commented on YARN-6964:

Previously the code was:{code}    Resource desiredShare = Resources.min(policy.getResourceCalculator(),
        scheduler.getClusterResource(), getMinShare(), getDemand());{code}  We were already
taking the min.  What I changed here is that instead of allowing the min share to inflate
the the desired share for the non-dominant resource, we take a proper component-wise min.
 Assume min share is 6GB and 6 vcores, and my demand is 6GB and 3 vcores.  If memory is dominant,
then the code before would boost the desired share to 6GB and 6 vcores.  With my change, the
desired share will be 6GB and 3 vcores.

Also note that this calculation is only for the purposes of determining whether the queue
is min share starved.  It has nothing to do with what the container will be assigned.

> Fair scheduler misuses Resources operations
> -------------------------------------------
>                 Key: YARN-6964
>                 URL: https://issues.apache.org/jira/browse/YARN-6964
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 3.0.0-alpha4
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>         Attachments: YARN-6964.001.patch
> There are several places where YARN uses the {{Resources}} class to do comparisons of
{{Resource}} instances incorrectly.  This patch corrects those mistakes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message