hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6610) DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate
Date Tue, 15 Aug 2017 23:51:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128098#comment-16128098
] 

Wangda Tan commented on YARN-6610:
----------------------------------

[~templedf], I just checked, in Java, Arrays.sort uses dual pivot quicksort, in implementation,
it actually use insertion sort for tiny arrays. (See http://codeblab.com/wp-content/uploads/2009/09/DualPivotQuicksort.pdf).

So I think the sort may not cause significant performance regression comparing to the original
approach.

If any regression happens, I would suggest to take a look at the additional array allocation
as well (even if JVM manages memory allocation pretty fast). Probably we can use ThreadLocal
static arrays to hold resource arrays instead of allocating arrays every time.

> DominantResourceCalculator.getResourceAsValue() dominant param is no longer appropriate
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-6610
>                 URL: https://issues.apache.org/jira/browse/YARN-6610
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: YARN-3926
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>            Priority: Critical
>         Attachments: YARN-6610.001.patch, YARN-6610.YARN-3926.002.patch, YARN-6610.YARN-3926.003.patch,
YARN-6610.YARN-3926.004.patch, YARN-6610.YARN-3926.005.patch, YARN-6610.YARN-3926.006.patch
>
>
> The {{dominant}} param assumes there are only two resources, i.e. true means to compare
the dominant, and false means to compare the subordinate.  Now that there are _n_ resources,
this parameter no longer makes sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message