hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5262) Allow specifying min shares as percentage of cluster
Date Sat, 14 Feb 2009 15:19:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673517#action_12673517

Hemanth Yamijala commented on HADOOP-5262:

+1. This would be a great addition. However, are you suggesting that we support both modes
? So, if specified without a '%', this would become an absolute number, and otherwise it would
be a percentage ?

Expressing it purely as percentages has a very simple operational advantage that it can be
validated. For e.g. if the percentages add up to more than 100, then we could warn the administrators
or even stop the scheduler. Allowing both modes would probably prevent us from such validations,
no ?

> Allow specifying min shares as percentage of cluster
> ----------------------------------------------------
>                 Key: HADOOP-5262
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5262
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: contrib/fair-share
>            Reporter: Matei Zaharia
>            Priority: Minor
> Currently the guaranteed shares for pools in the fair scheduler are specified as a number
of slots. For organizations where a group pays X% of the cluster and the actual number of
nodes in the cluster varies due to failures, expansion, etc over time, it would be useful
to support a guaranteed share given as a percentage too. This would just let you write in
the config file something like <minMaps>5%</minMaps> instead of <minMaps>42</minMaps>.
The scheduler would need to recompute what this means in terms of number of slots on every
update (probably through some kind of update(ClusterStatus) method in PoolManager).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message