hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1495) Allow moving apps between queues
Date Tue, 11 Feb 2014 01:36:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897421#comment-13897421

Hitesh Shah commented on YARN-1495:

The way I see it - it is and should be ok for different schedulers to support a different
set of features. The behavior should be the same across all schedulers if the feature is supported.

@Karthik, I dont believe it is right to do a half-baked approach regardless of which scheduler
builds the feature first. The main concern is for an app developer and how a new feature or
the lack of it affects someone writing their own application. 

>From an API point of view, there should be a way for the application at run-time/registration
time to find out what features are supported or not supported by the currently configured
scheduler in the RM. This allows for applications to be written correctly and to make the
necessary changes in the calls to the RM to work around advanced vs primitive schedulers.
If schedulers are going to differ in terms of feature support, then an API to find out whether
a feature is supported or not should be considered a blocker for a release. I believe this
only holds for APIs affecting application masters for now but there may be situations where
a client could be affected too.

Now, for moving apps across schedulers - given that it is a client only feature and there
is no changes required in an application, my previous comment's argument does not hold for
this feature. ( I assume that Fifo and CS will both throw an appropriate UnsupportedOperationException
on a move call? ) 

> Allow moving apps between queues
> --------------------------------
>                 Key: YARN-1495
>                 URL: https://issues.apache.org/jira/browse/YARN-1495
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: scheduler
>    Affects Versions: 2.2.0
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
> This is an umbrella JIRA for work needed to allow moving YARN applications from one queue
to another.  The work will consist of additions in the command line options, additions in
the client RM protocol, and changes in the schedulers to support this.
> I have a picture of how this should function in the Fair Scheduler, but I'm not familiar
enough with the Capacity Scheduler for the same there.  Ultimately, the decision to whether
an application can be moved should go down to the scheduler - some schedulers may wish not
to support this at all.  However, schedulers that do support it should share some common semantics
around ACLs and what happens to running containers.
> Here is how I see the general semantics working out:
> * A move request is issued by the client.  After it gets past ACLs, the scheduler checks
whether executing the move will violate any constraints. For the Fair Scheduler, these would
be queue maxRunningApps and queue maxResources constraints
> * All running containers are transferred from the old queue to the new queue
> * All outstanding requests are transferred from the old queue to the new queue
> Here is I see the ACLs of this working out:
> * To move an app from a queue a user must have modify access on the app or administer
access on the queue
> * To move an app to a queue a user must have submit access on the queue or administer
access on the queue 

This message was sent by Atlassian JIRA

View raw message