hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5139) [Umbrella] Move YARN scheduler towards global scheduler
Date Tue, 24 May 2016 21:45:12 GMT
Wangda Tan created YARN-5139:
--------------------------------

             Summary: [Umbrella] Move YARN scheduler towards global scheduler
                 Key: YARN-5139
                 URL: https://issues.apache.org/jira/browse/YARN-5139
             Project: Hadoop YARN
          Issue Type: New Feature
            Reporter: Wangda Tan
            Assignee: Wangda Tan


Existing YARN scheduler is based on node heartbeat. This can lead to sub-optimal decisions
because scheduler can only look at one node at the time when scheduling resources.

Pseudo code of existing scheduling logic looks like:
{code}
for node in allNodes:
   Go to parentQueue
      Go to leafQueue
        for application in leafQueue.applications:
           for resource-request in application.resource-requests
              try to schedule on node
{code}

Considering future complex resource placement requirements, such as node constraints (give
me "a && b || c") or anti-affinity (do not allocate HBase regionsevers and Storm workers
on the same host), we may need to consider moving YARN scheduler towards global scheduling.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message