hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6418) Reduce Resource object creation and overhead in Capacity scheduler inner loop
Date Fri, 31 Mar 2017 18:28:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951451#comment-15951451

Wangda Tan commented on YARN-6418:


I have done some similar works in YARN-6245 while working on the scheduler perf profiling
before, please see the patch: https://issues.apache.org/jira/secure/attachment/12854991/YARN-6245.preliminary-staled.1.patch.

Please let me know if we're talking about the same thing. I suggest to move this JIRA to YARN-6242
for better tracking.

> Reduce Resource object creation and overhead in Capacity scheduler inner loop
> -----------------------------------------------------------------------------
>                 Key: YARN-6418
>                 URL: https://issues.apache.org/jira/browse/YARN-6418
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler, resourcemanager
>    Affects Versions: 2.7.3, 3.0.0-alpha2, 2.8
>            Reporter: Roni Burd
>            Assignee: Roni Burd
> Resource object is used multiple due to ResourceCalculator creates new instances on every
method call. This gets called several times on each node HB. Resource is a very expensive
object that relies on Protobufs 
> The change is to remove the need to use protobuf on the Resource object and avoid creating
many objects in the Resource Calculator all the time

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message