hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-6299) FairSharePolicy is incorrect when demand is less than min share
Date Sat, 11 Mar 2017 00:20:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905916#comment-15905916
] 

Daniel Templeton edited comment on YARN-6299 at 3/11/17 12:19 AM:
------------------------------------------------------------------

My concern is that if I have two apps that are both under min share, we will prefer the app
that has used fewer of the resources assigned to it.  For example, assume min share is 16GB.
 Assume app1 has requested 10GB and used 4GB.  app2 has also requested 5GB and used 4GB.

{code}
resourceUsage1 = 4
resourceUsage2 = 4
minShare1 = 10
minShare2 = 5
s1Needy = true
s2Needy = true
minshareRatio1 = 0.4
minshareRatio2 = 0.8
{code}

We will therefore return {{0.4 - 0.8}}, which is negative, meaning that we prefer app1 to
app2.  Unless I'm misunderstanding something.


was (Author: templedf):
My concern is that if I have two apps that are both under min share, we will prefer the app
that has used fewer of the resources assigned to it.  For example, assume min share is 16GB.
 Assume app1 has requested 10GB and used 4GB.  app2 has also requested 10GB and used 8GB.

{code}
resourceUsage1 = 4
resourceUsage2 = 8
minShare1 = 10
minShare2 = 10
s1Needy = true
s2Needy = true
minshareRatio1 = 0.4
minshareRatio2 = 0.8
{code}

We will therefore return {{0.4 - 0.8}}, which is negative, meaning that we prefer app1 to
app2.  Unless I'm misunderstanding something.

> FairSharePolicy is incorrect when demand is less than min share
> ---------------------------------------------------------------
>
>                 Key: YARN-6299
>                 URL: https://issues.apache.org/jira/browse/YARN-6299
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Daniel Templeton
>
> {code}
>       Resource resourceUsage1 = s1.getResourceUsage();
>       Resource resourceUsage2 = s2.getResourceUsage();
>       Resource minShare1 = Resources.min(RESOURCE_CALCULATOR, null,
>           s1.getMinShare(), s1.getDemand());
>       Resource minShare2 = Resources.min(RESOURCE_CALCULATOR, null,
>           s2.getMinShare(), s2.getDemand());
>       boolean s1Needy = Resources.lessThan(RESOURCE_CALCULATOR, null,
>           resourceUsage1, minShare1);
>       boolean s2Needy = Resources.lessThan(RESOURCE_CALCULATOR, null,
>           resourceUsage2, minShare2);
>       minShareRatio1 = (double) resourceUsage1.getMemorySize()
>           / Resources.max(RESOURCE_CALCULATOR, null, minShare1, ONE).getMemorySize();
>       minShareRatio2 = (double) resourceUsage2.getMemorySize()
>           / Resources.max(RESOURCE_CALCULATOR, null, minShare2, ONE).getMemorySize();
> {code}
> If demand is less than min share, then an app will be flagged as needy if it has demand
that is higher than its usage, which happens any time the app has been assigned resources
that it hasn't started using yet.  That sounds wrong to me.  [~kasha], [~yufeigu]?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message