Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 21 Apr 2016 23:21:13 +0000 (UTC)
From: "Karthik Kambatla (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12673386.1381477083000.2858.1461280873119@Atlassian.JIRA>
In-Reply-To: <JIRA.12673386.1381477083000@Atlassian.JIRA>
References: <JIRA.12673386.1381477083000@Atlassian.JIRA>
 <JIRA.12673386.1381477083524@arcas>
Subject: [jira] [Commented] (YARN-1297) Miscellaneous Fair Scheduler
 speedups
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252984#comment-15252984 ] 

Karthik Kambatla commented on YARN-1297:
----------------------------------------

Looks like the test failures are because of preemption and runnability logic and test's reliance on the timing of the queue-usage update. I remember Sandy measured the logging changes themselves led to good improvement. 

What do you think of doing only the logging changes here and drive the resources change as part of YARN-4090? 

> Miscellaneous Fair Scheduler speedups
> -------------------------------------
>
>                 Key: YARN-1297
>                 URL: https://issues.apache.org/jira/browse/YARN-1297
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>            Reporter: Sandy Ryza
>            Assignee: Yufei Gu
>         Attachments: YARN-1297-1.patch, YARN-1297-2.patch, YARN-1297.005.patch, YARN-1297.3.patch, YARN-1297.4.patch, YARN-1297.4.patch, YARN-1297.patch, YARN-1297.patch
>
>
> I ran the Fair Scheduler's core scheduling loop through a profiler tool and identified a bunch of minimally invasive changes that can shave off a few milliseconds.
> The main one is demoting a couple INFO log messages to DEBUG, which brought my benchmark down from 16000 ms to 6000.
> A few others (which had way less of an impact) were
> * Most of the time in comparisons was being spent in Math.signum.  I switched this to direct ifs and elses and it halved the percent of time spent in comparisons.
> * I removed some unnecessary instantiations of Resource objects
> * I made it so that queues' usage wasn't calculated from the applications up each time getResourceUsage was called.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)