hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Shao (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3806) Proposal of Generic Scheduling Framework for YARN
Date Tue, 16 Jun 2015 03:57:00 GMT
Wei Shao created YARN-3806:

             Summary: Proposal of Generic Scheduling Framework for YARN
                 Key: YARN-3806
                 URL: https://issues.apache.org/jira/browse/YARN-3806
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: scheduler
            Reporter: Wei Shao

Currently, A typical YARN cluster runs many different kinds of applications: production applications,
ad hoc user applications, long running services and so on. Different YARN scheduling policies
may be suitable for different applications. For example, capacity scheduling can manage production
applications well since application can get guaranteed resource share, fair scheduling can
manage ad hoc user applications well since it can enforce fairness among users. However, current
YARN scheduling framework doesn’t have a mechanism for multiple scheduling policies work
hierarchically in one cluster.

YARN-3306 talked about many issues of today’s YARN scheduling framework, and proposed a
per-queue policy driven framework. In detail, it supported different scheduling policies for
leaf queues. However, support of different scheduling policies for upper level queues is not
seriously considered yet. 

A generic scheduling framework is proposed here to address these limitations. It supports
different policies for any queue consistently. The proposal tries to solve many other issues
in current YARN scheduling framework as well.

This message was sent by Atlassian JIRA

View raw message