hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2986) Support hierarchical and unified scheduler configuration
Date Tue, 23 Dec 2014 18:12:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-2986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257288#comment-14257288
] 

Vinod Kumar Vavilapalli commented on YARN-2986:
-----------------------------------------------

Today's scheduler configuration is fragmented and non-intuitive, and needs to be improved.
 - Each scheduler has its own configuration file and schema
 - Each scheduler maintains its own custom configuration parser
 - Though there is a first-class notion of many things, the configurations are duplicated.
For e.g queues are first-class, but configuration queue-acls is per scheduler
 - Scheduler configuration is split across yarn-site.xml and capacity-scheduler/fair-scheduler
files
 - Queues have a notion of hierarchy, but the configuration is a flat dictionary. This is
very painful to configure

It'll be great to have 
 - a single configuration file that configures everything related to scheduling
 - a single file with plugin points for individual schedulers
 - coupled with a single parser with custom parse-points for individual schedulers
 - configuration file follow a hierarchical schema.

Note that none of this is new. MAPREDUCE-861 originally did this in Hadoop-1, but it got lost
during the rewrite that is YARN. This JIRA proposes to bring all of it back.

> Support hierarchical and unified scheduler configuration
> --------------------------------------------------------
>
>                 Key: YARN-2986
>                 URL: https://issues.apache.org/jira/browse/YARN-2986
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>
> Today's scheduler configuration is fragmented and non-intuitive, and needs to be improved.
Details in comments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message