hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5139) [Umbrella] Move YARN scheduler towards global scheduler
Date Tue, 24 May 2016 21:45:12 GMT
Wangda Tan created YARN-5139:

             Summary: [Umbrella] Move YARN scheduler towards global scheduler
                 Key: YARN-5139
                 URL: https://issues.apache.org/jira/browse/YARN-5139
             Project: Hadoop YARN
          Issue Type: New Feature
            Reporter: Wangda Tan
            Assignee: Wangda Tan

Existing YARN scheduler is based on node heartbeat. This can lead to sub-optimal decisions
because scheduler can only look at one node at the time when scheduling resources.

Pseudo code of existing scheduling logic looks like:
for node in allNodes:
   Go to parentQueue
      Go to leafQueue
        for application in leafQueue.applications:
           for resource-request in application.resource-requests
              try to schedule on node

Considering future complex resource placement requirements, such as node constraints (give
me "a && b || c") or anti-affinity (do not allocate HBase regionsevers and Storm workers
on the same host), we may need to consider moving YARN scheduler towards global scheduling.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message