hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-6775) CapacityScheduler: Improvements to assignContainers()
Date Fri, 07 Jul 2017 16:43:00 GMT
Nathan Roberts created YARN-6775:
------------------------------------

             Summary: CapacityScheduler: Improvements to assignContainers()
                 Key: YARN-6775
                 URL: https://issues.apache.org/jira/browse/YARN-6775
             Project: Hadoop YARN
          Issue Type: Sub-task
          Components: capacityscheduler
    Affects Versions: 3.0.0-alpha3, 2.8.1
            Reporter: Nathan Roberts
            Assignee: Nathan Roberts


There are several things in assignContainers() that are done multiple times even though the
result cannot change (canAssignToUser, canAssignToQueue). Add some local caching to take advantage
of this fact.

Will post patch shortly. Patch includes a simple throughput test that demonstrates when we
have users at their user-limit, the number of NodeUpdateSchedulerEvents we can process can
be improved from 13K/sec to 50K/sec.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message