hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3983) Make CapacityScheduler to easier extend application allocation logic
Date Mon, 27 Jul 2015 21:35:05 GMT
Wangda Tan created YARN-3983:
--------------------------------

             Summary: Make CapacityScheduler to easier extend application allocation logic
                 Key: YARN-3983
                 URL: https://issues.apache.org/jira/browse/YARN-3983
             Project: Hadoop YARN
          Issue Type: Bug
            Reporter: Wangda Tan
            Assignee: Wangda Tan


While working on YARN-1651 (resource allocation for increasing container), I found it is very
hard to extend existing CapacityScheduler resource allocation logic to support different types
of resource allocation.

For example, there's a lot of differences between increasing a container and allocating a
container:
- Increasing a container doesn't need to check locality delay.
- Increasing a container doesn't need to build/modify a resource request tree (ANY->RACK/HOST).
- Increasing a container doesn't need to check allocation/reservation starvation (see {{shouldAllocOrReserveNewContainer}}).
- After increasing a container is approved by scheduler, it need to update an existing container
token instead of creating new container.

And there're lots of similarities when allocating different types of resources.
- User-limit/queue-limit will be enforced for both of them.
- Both of them needs resource reservation logic. (Maybe continuous reservation looking is
needed for both of them).

The purpose of this JIRA is to make easier extending CapacityScheduler resource allocation
logic to support different types of resource allocation, make common code reusable, and also
better code organization.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message