hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6788) Improve performance of resource profile branch
Date Wed, 19 Jul 2017 19:18:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sunil G updated YARN-6788:
--------------------------
    Attachment: YARN-6788-YARN-3926.007.patch

Fixed all comments from [~daniel@cloudera.com] with below points open.

bq.In Resource.compareTo(), array length is an int, so diff should be an int.
We use long for getMemorySize. Hence i had to use long here.

bq.Seems like you should add a ResourceUtils.getResourceType(String resource) method to
simplify the code
I kind of optimized use of ResourceUtils for iteration only. Very few random access needed
now. 

bq.In the BaseResource() constructor, I don't see a reason to special case the memory and
CPU
Optimized BaseResource for CPU and mem so that we can push for performance as possible. In
light weight mode, this saves a lot performance impact. For multi resources as well, we can
use PBImpl for now.


Lastly, there were some more performance gaps which is addressed in this patch. I ll share
another version of patch post fixing and verifying all test cases locally.


> Improve performance of resource profile branch
> ----------------------------------------------
>
>                 Key: YARN-6788
>                 URL: https://issues.apache.org/jira/browse/YARN-6788
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>            Priority: Blocker
>         Attachments: YARN-6788-YARN-3926.001.patch, YARN-6788-YARN-3926.002.patch, YARN-6788-YARN-3926.003.patch,
YARN-6788-YARN-3926.004.patch, YARN-6788-YARN-3926.005.patch, YARN-6788-YARN-3926.006.patch,
YARN-6788-YARN-3926.007.patch
>
>
> Currently we could see a 15% performance delta with this branch. 
> Few performance improvements to improve the same.
> Also this patch will handle [comments|https://issues.apache.org/jira/browse/YARN-6761?focusedCommentId=16075418&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16075418]
from [~leftnoteasy].



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message