Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <22577236.272181290547697208.JavaMail.jira@thor>
Date: Tue, 23 Nov 2010 16:28:17 -0500 (EST)
From: "Scott Chen (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-2198) Allow FairScheduler to control
 the number of slots on each TaskTracker
In-Reply-To: <7372674.250691290480014352.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935052#action_12935052 ] 

Scott Chen commented on MAPREDUCE-2198:
---------------------------------------

Hey Arun,

bq. You also have to worry about piggy-backing of task-cleanup tasks done by the JT...
I read some codes. Now I see your point.
JobTracker.getSetupAndCleanupTasks() is not controlled by the Scheduler.
So we need to put some logic there to make it aware of this task limit. Or we might be over-scheduled.

> Allow FairScheduler to control the number of slots on each TaskTracker
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2198
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2198
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/fair-share
>    Affects Versions: 0.22.0
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>
>
> We can set the number of slots on the TaskTracker to be high and let FairScheduler handles the slots.
> This approach allows us to change the number of slots on each node dynamically.
> The administrator can change the number of slots with a CLI tool.
> One use case of this is for upgrading the MapReduce.
> Instead of restarting the cluster, we can run the new MapReduce on the same cluster.
> And use the CLI tool to gradually migrate the slots.
> This way we don't lost the progress fo the jobs that's already executed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.