Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1950407720.1245375307337.JavaMail.jira@brutus>
Date: Thu, 18 Jun 2009 18:35:07 -0700 (PDT)
From: "Namit Jain (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-6082) Elegant decommission of lighty
 loaded tasktrackers from a map-reduce cluster
In-Reply-To: <150681110.1245371887463.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-6082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721607#action_12721607 ] 

Namit Jain commented on HADOOP-6082:
------------------------------------

The JobTracker needs a new API:

something like: 

decommissionTaskTrackers(int numberOfTT);

The TaskTracker may need to expose a new API to return its current load.

The JobTracker will get the current load of each TaskTracker and then decide to decommission the
most lightly loaded 'n' takstrackers.

When a TaskTracker is being decommissioned, it will stop accepting new jobs, and will
die when all the current jobs are finished. This may lead to wastage of resources in the cluster.

The jobtracker can optionally pass a timeout after which the tasktracker will definitely die.
At that time, it might be a good idea to increase the number of retires for the tasks being
executed.

The UI may neeed to be changed to show the new status of the task tracker as well.

> Elegant decommission of lighty loaded tasktrackers from a map-reduce cluster
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-6082
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6082
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: dhruba borthakur
>            Assignee: Namit Jain
>
> There is a need to elegantly move some machines from one map-reduce cluster to another. This JIRA is to discuss how to find lightly loaded tasktrackers that are candidates for decommissioning and then to elegantly decommission them by waiting for existing tasks to finish.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.