hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6788) Improve performance of resource profile branch
Date Tue, 01 Aug 2017 22:35:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109913#comment-16109913
] 

Daniel Templeton commented on YARN-6788:
----------------------------------------

Thanks for the updated patch, [~sunilg].  Here are the issues I still see open:

* In {{Resources.addTo()}}, {{Resources.subtractFrom()}}, {{Resources.multiplyTo()}}, {{Resources.multiplyAndAddTo()}},
{{Resources.multiplyAndRoundDown()}}, {{Resources.fitsIn()}}, {{Resources.componentwiseMin()}},
and {{Resources.componentwiseMax()}}, the variable in the _foreach_ should be named {{lhsValue}}
instead of calling it {{entry}} and then declaring a new variable for it called {{lhsValue}}.
* {quote}In DominantResourceCalculator(), we already invoke getResourceTypes. So i think its
fine.{quote}  In {{ResourceUtils}} you're creating an API, and there's no reason some other
part of the code wouldn't call {{getResourceNamesArray()}} in the future.  To prevent nasty
surprises, the methods of your API should have consistent behavior.  If you're worried about
performance cost of the initialization check, then you should take a different approach. 
See my next point.
* The DCL still won't work as implemented.  DCL is hard to get right, and it's with good reason
that it's suggested to follow a recipe for DCL with no modifications.  Instead, how about
making the {{ResourceUtils}} class into a proper singleton?  You can then do the initialization
in the {{getInstance()}} call and not have to worry about checking it in every API call. 
Yeah, you'll still need DCL for the initialization, but with a singleton you'll be able to
use a DCL recipe unaltered.

> Improve performance of resource profile branch
> ----------------------------------------------
>
>                 Key: YARN-6788
>                 URL: https://issues.apache.org/jira/browse/YARN-6788
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>            Priority: Blocker
>         Attachments: YARN-6788-YARN-3926.001.patch, YARN-6788-YARN-3926.002.patch, YARN-6788-YARN-3926.003.patch,
YARN-6788-YARN-3926.004.patch, YARN-6788-YARN-3926.005.patch, YARN-6788-YARN-3926.006.patch,
YARN-6788-YARN-3926.007.patch, YARN-6788-YARN-3926.008.patch, YARN-6788-YARN-3926.009.patch,
YARN-6788-YARN-3926.010.patch, YARN-6788-YARN-3926.011.patch, YARN-6788-YARN-3926.012.patch,
YARN-6788-YARN-3926.013.patch, YARN-6788-YARN-3926.014.patch, YARN-6788-YARN-3926.015.patch,
YARN-6788-YARN-3926.016.patch, YARN-6788-YARN-3926.017.patch, YARN-6788-YARN-3926.018.patch,
YARN-6788-YARN-3926.019.patch
>
>
> Currently we could see a 15% performance delta with this branch. 
> Few performance improvements to improve the same.
> Also this patch will handle [comments|https://issues.apache.org/jira/browse/YARN-6761?focusedCommentId=16075418&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16075418]
from [~leftnoteasy].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message