hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7005) Skip unnecessary sorting and iterating process for child queues without pending resource to optimize schedule performance
Date Tue, 12 Dec 2017 00:42:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286868#comment-16286868
] 

Wangda Tan edited comment on YARN-7005 at 12/12/17 12:41 AM:
-------------------------------------------------------------

[~Tao Yang], apologize for my late responses, a better way to test might be by adding unit
test to TestCapacitySchedulerPerf. Which is portable and easy for other folks to reproduce
your results. SLS could be also used when more comprehensive tests are required. 


was (Author: leftnoteasy):
[~Tao Yang], apologize for my late responses, a better way to test might be by using TestCapacitySchedulerPerf.
Which is portable and easy for other folks to reproduce your results. SLS could be also used
when more comprehensive tests are required. 

> Skip unnecessary sorting and iterating process for child queues without pending resource
to optimize schedule performance
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-7005
>                 URL: https://issues.apache.org/jira/browse/YARN-7005
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.9.0, 3.0.0-alpha4
>            Reporter: Tao Yang
>         Attachments: YARN-7005.001.patch
>
>
> Nowadays even if there is only one pending app in a queue, the scheduling process will
go through all queues anyway and costs most of time on sorting and iterating child queues
in ParentQueue#assignContainersToChildQueues. 
> IIUIC, queues that have no pending resource can be skipped for sorting and iterating
process to reduce time cost, obviously for a cluster with many queues. Please feel free to
correct me if I ignore something else. Thanks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message