hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1495) Allow moving apps between queues
Date Fri, 07 Feb 2014 20:00:23 GMT

    [ https://issues.apache.org/jira/browse/YARN-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13894945#comment-13894945
] 

Hitesh Shah commented on YARN-1495:
-----------------------------------

Pre-emption is an internal implementation choice of how a scheduler enforces its policies.
This case is more towards supporting a feature that users can use - but users would then be
forced to be aware of what scheduler is being used to decide whether the feature is supported
or not.

It is good that schedulers are getting enhanced however it seems wrong from a YARN user or
app-developer's point of view if the user has to do things differently just because a scheduler
does/does not support a feature. There should be a way to define an API for the app developer
which ensures that there is a strict contract on what will be done. There could be some way
to define implementation-specific APIs too but those should be clearly called out that the
feature/api is not supported by all schedulers.




> Allow moving apps between queues
> --------------------------------
>
>                 Key: YARN-1495
>                 URL: https://issues.apache.org/jira/browse/YARN-1495
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: scheduler
>    Affects Versions: 2.2.0
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>
> This is an umbrella JIRA for work needed to allow moving YARN applications from one queue
to another.  The work will consist of additions in the command line options, additions in
the client RM protocol, and changes in the schedulers to support this.
> I have a picture of how this should function in the Fair Scheduler, but I'm not familiar
enough with the Capacity Scheduler for the same there.  Ultimately, the decision to whether
an application can be moved should go down to the scheduler - some schedulers may wish not
to support this at all.  However, schedulers that do support it should share some common semantics
around ACLs and what happens to running containers.
> Here is how I see the general semantics working out:
> * A move request is issued by the client.  After it gets past ACLs, the scheduler checks
whether executing the move will violate any constraints. For the Fair Scheduler, these would
be queue maxRunningApps and queue maxResources constraints
> * All running containers are transferred from the old queue to the new queue
> * All outstanding requests are transferred from the old queue to the new queue
> Here is I see the ACLs of this working out:
> * To move an app from a queue a user must have modify access on the app or administer
access on the queue
> * To move an app to a queue a user must have submit access on the queue or administer
access on the queue 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message