Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Mon, 22 Feb 2016 18:08:18 +0000 (UTC)
From: "Ming Ma (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12938732.1455260829000.111809.1456164498129@Atlassian.JIRA>
In-Reply-To: <JIRA.12938732.1455260829000@Atlassian.JIRA>
References: <JIRA.12938732.1455260829000@Atlassian.JIRA>
 <JIRA.12938732.1455260829173@arcas>
Subject: [jira] [Commented] (YARN-4690) Skip object allocation in
 FSAppAttempt#getResourceUsage when possible
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157397#comment-15157397 ] 

Ming Ma commented on YARN-4690:
-------------------------------

Thanks [~sjlee0] and [~kasha]!

> Skip object allocation in FSAppAttempt#getResourceUsage when possible
> ---------------------------------------------------------------------
>
>                 Key: YARN-4690
>                 URL: https://issues.apache.org/jira/browse/YARN-4690
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>             Fix For: 2.8.0, 2.7.3, 2.9.0, 2.6.5
>
>         Attachments: YARN-4690.patch
>
>
> YARN-2768 addresses an important bottleneck. Here is another similar instance where object allocation in Resources#subtract will slow down the fair scheduler's event processing thread.
> {noformat}
> org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java)
>     org.apache.hadoop.yarn.util.Records.newRecord(Records.java)
>     org.apache.hadoop.yarn.util.resource.Resources.createResource(Resources.java)
>     org.apache.hadoop.yarn.util.resource.Resources.clone(Resources.java)
>     org.apache.hadoop.yarn.util.resource.Resources.subtract(Resources.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt.getResourceUsage(FSAppAttempt.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.getResourceUsage(FSLeafQueue.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.FairSharePolicy$FairShareComparator.compare(FairSharePolicy.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.FairSharePolicy$FairShareComparator.compare(FairSharePolicy.java)
>     java.util.TimSort.binarySort(TimSort.java)
>     java.util.TimSort.sort(TimSort.java)
>     java.util.TimSort.sort(TimSort.java)
>     java.util.Arrays.sort(Arrays.java)
>     java.util.Collections.sort(Collections.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.assignContainer(FSParentQueue.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java)
>     org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java)
>     org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.handle(ResourceSchedulerWrapper.java)
>     org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.handle(ResourceSchedulerWrapper.java)
> {noformat}
> One way to fix it is to return {{getCurrentConsumption()}} if there is no preemption which is the normal case. This means {{getResourceUsage}} method will return reference to {{FSAppAttempt}}'s internal resource object. But that should be ok as {{getResourceUsage}} doesn't expect the caller to modify the object.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)