hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5262) Allow specifying min shares as percentage of cluster
Date Tue, 17 Feb 2009 04:14:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674094#action_12674094

Hemanth Yamijala commented on HADOOP-5262:

+1 for a more passive load of the configuration.
+1 also for logging warnings on runtime errors. IMO, it is reasonable to have a place to display
events that need attention on the web ui or have some other alerting option, and this is not
just for the scheduler.

One more suggestion we've had from our Operations team was to provide a validator tool that
can validate configuration (given a conf file, for instance), before it can be deployed on
the cluster.

These all make sense, but I believe are a subject of different JIRAs.

> Allow specifying min shares as percentage of cluster
> ----------------------------------------------------
>                 Key: HADOOP-5262
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5262
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: contrib/fair-share
>            Reporter: Matei Zaharia
>            Priority: Minor
> Currently the guaranteed shares for pools in the fair scheduler are specified as a number
of slots. For organizations where a group pays X% of the cluster and the actual number of
nodes in the cluster varies due to failures, expansion, etc over time, it would be useful
to support a guaranteed share given as a percentage too. This would just let you write in
the config file something like <minMaps>5%</minMaps> instead of <minMaps>42</minMaps>.
The scheduler would need to recompute what this means in terms of number of slots on every
update (probably through some kind of update(ClusterStatus) method in PoolManager).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message