Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1041219544.1233955619668.JavaMail.jira@brutus>
Date: Fri, 6 Feb 2009 13:26:59 -0800 (PST)
From: "Matei Zaharia (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-5186) Improve limit handling in fairshare
 scheduler
In-Reply-To: <1695864363.1233919199731.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671327#action_12671327 ] 

Matei Zaharia commented on HADOOP-5186:
---------------------------------------

Both suggestions make a lot of sense. So, to clarify, would the logic look like this?
- For each pool, initialize and schedule at least its maxRunningJobs number of jobs.
- If all tasks from all initialized jobs in the pool are running, then initialize further jobs.

> Improve limit handling in fairshare scheduler
> ---------------------------------------------
>
>                 Key: HADOOP-5186
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5186
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Hemanth Yamijala
>            Priority: Minor
>
> The fairshare scheduler has a way by which it can limit the number of jobs in a pool by setting the maxRunningJobs parameter in its allocations definition. This limit is treated as a hard limit, and comes into effect even if the cluster is free to run more jobs, resulting in underutilization. Possibly the same thing happens with the parameter maxRunningJobs for user and userMaxJobsDefault. It may help to treat these as a soft limit and run additional jobs to keep the cluster fully utilized.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.