hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6610) DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate
Date Wed, 16 Aug 2017 21:47:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wangda Tan updated YARN-6610:
    Attachment: YARN-6610.YARN-3926.perf-wangda-001.patch

Investigated more, it looks like original perf patch has some typos inside {{calculateShares2}},
I tried to run all CapacityScheduler tests with the old perf patch, it causes many failures.

With attached patch {{YARN-6610.YARN-3926.perf-wangda-001.patch}}, it has 2 failures, but
looks unrelated. (Ran unit test w/o the patch, it fails as well).

[~templedf], it looks like existing tests in TestResourceCalculator are not enough to capture
the issue, I tried to add some (testCompare2) but it looks not enough as well. If you have
any cycles, could you help to add more unit tests?

I also tried to run testUserLimitThroughput again, the latest patch shows very similar result.

Since this is blocker of YARN-3926 merge, and it doesn't show performance regression, I plan
to commit this patch to branch by end of today if no opposite opinions.

> DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate
> ---------------------------------------------------------------------------------------
>                 Key: YARN-6610
>                 URL: https://issues.apache.org/jira/browse/YARN-6610
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: YARN-3926
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>            Priority: Critical
>         Attachments: YARN-6610.001.patch, YARN-6610.YARN-3926.002.patch, YARN-6610.YARN-3926.003.patch,
YARN-6610.YARN-3926.004.patch, YARN-6610.YARN-3926.005.patch, YARN-6610.YARN-3926.006.patch,
YARN-6610.YARN-3926.006-rebased.patch, YARN-6610.YARN-3926.perf.patch, YARN-6610.YARN-3926.perf-rebased.patch,
> The {{dominant}} param assumes there are only two resources, i.e. true means to compare
the dominant, and false means to compare the subordinate.  Now that there are _n_ resources,
this parameter no longer makes sense.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message