Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 1 Jun 2017 20:48:04 +0000 (UTC)
From: "Daryn Sharp (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13076572.1496332895000.344958.1496350084259@Atlassian.JIRA>
In-Reply-To: <JIRA.13076572.1496332895000@Atlassian.JIRA>
References: <JIRA.13076572.1496332895000@Atlassian.JIRA> <JIRA.13076572.1496332895129@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-6679) Reduce Resource instance overhead
 via non-PBImpl
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 01 Jun 2017 20:48:08 -0000


    [ https://issues.apache.org/jira/browse/YARN-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033655#comment-16033655 ] 

Daryn Sharp commented on YARN-6679:
-----------------------------------

Thanks Daniel!

Using signum to handle a NaN (impossible, I hope?) may not be worth the cost.   I only moved the pre-existing method from the pb impl to the base class so I'd suggest another jira if feel strongly that it should be changed?

bq. How thoroughly has this been tested?
I'd say very.  The first large/busy pre-production cluster was crippled by 2.8.  The scheduler thread was constantly pegging a cpu and falling behind.  Deploys were halted.  We deployed my collection of patches about 1.5w ago.  Cpu fluctuates a lot, but doesn't stay pegged anymore.

bq. Wanna add some unit tests to confirm the newInstance() methods and the PB conversion work as expected?
I would certainly hope the existing tests provide coverage! :)  I didn't expose any new methods to test but I'll concoct some rudimentary tests if need be.

> Reduce Resource instance overhead via non-PBImpl
> ------------------------------------------------
>
>                 Key: YARN-6679
>                 URL: https://issues.apache.org/jira/browse/YARN-6679
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.8.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: YARN-6679.branch-2.patch, YARN-6679.trunk.patch
>
>
> Creating and using transient PB-based Resource instances during scheduling is very expensive.  The overhead can be transparently reduced by internally using lightweight non-PB based instances.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org