hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6788) Improve performance of resource profile branch
Date Tue, 18 Jul 2017 17:40:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sunil G updated YARN-6788:
    Attachment: YARN-6788-YARN-3926.006.patch

Latest patch addresses test failures and its more a cleaner patch.

Attaching an initial test report also. I will attach a more detailed doc with graph and other
test cases sooner. cc/[~leftnoteasy] [~vvasudev] and [~templedf]

*Test Setup (SLS)*
Number of simulated node manager = 8000
Node Manager Memory = 128G
Number of applications submitted =  435 applications at the beginning of the test
Size of container ranged from 1G to 8G
Life time of containers are ranged from few secs to minutes.

*Test Case 1 (Basic resources: CPU and Memory)*
Use Dominant Resource Calculator to consider CPU and Memory. This test case is to compare
a direct performance between YARN-3926 and trunk on basic resources supported as of today
+Test Results+
Run SLS tests in 2 mins, throughput-per-second = #total-allocated-containers / 120.
Branch YARN-3926: Around 2200 containers / sec
Trunk: Around 2100 containers per / sec.

> Improve performance of resource profile branch
> ----------------------------------------------
>                 Key: YARN-6788
>                 URL: https://issues.apache.org/jira/browse/YARN-6788
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>            Priority: Blocker
>         Attachments: YARN-6788-YARN-3926.001.patch, YARN-6788-YARN-3926.002.patch, YARN-6788-YARN-3926.003.patch,
YARN-6788-YARN-3926.004.patch, YARN-6788-YARN-3926.005.patch, YARN-6788-YARN-3926.006.patch
> Currently we could see a 15% performance delta with this branch. 
> Few performance improvements to improve the same.
> Also this patch will handle [comments|https://issues.apache.org/jira/browse/YARN-6761?focusedCommentId=16075418&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16075418]
from [~leftnoteasy].

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message