hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-250) Add a generic mechanism to the resource manager for client communication with the scheduler.
Date Tue, 08 Oct 2013 20:54:43 GMT

    [ https://issues.apache.org/jira/browse/YARN-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789687#comment-13789687
] 

Sandy Ryza commented on YARN-250:
---------------------------------

The mechanism this JIRA proposed would make most sense for features specific to a single scheduler.
 As queue-moving functionality is probably something that the Capacity Scheduler would like
at some point as well, I think it would make more sense to directly add it as a ResourceManager
API.

> Add a generic mechanism to the resource manager for client communication with the scheduler.
> --------------------------------------------------------------------------------------------
>
>                 Key: YARN-250
>                 URL: https://issues.apache.org/jira/browse/YARN-250
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager, scheduler
>    Affects Versions: 2.0.2-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>
> In MR1 the fair scheduler allowed the queue of a running job to be changed through the
web UI.  For feature parity, this should be supported in MR2, but the web UI seems an inappropriate
place to put it because of complicated security implications and lack of programmatic access.
 A command line tool makes more sense.
>  
> A generic mechanism could be leveraged by other schedulers to support similar types of
functionality and would allow us to avoid making changes to all the plumbing each time functionality
is added.  Other possible uses might include suspending pools or fetching scheduler-specific
information.
>  
> This functionality could be made available through an RPC server within each scheduler,
but that would require reserving another port, and would introduce unnecessary confusion if
different schedulers implemented the same mechanism in a different way.
> A client should be able to send an RPC with a set of key/value pairs, which would be
passed to the scheduler.  The RPC would return a string (or set of key/value pairs).



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message