hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6082) Elegant decommission of lighty loaded tasktrackers from a map-reduce cluster
Date Fri, 19 Jun 2009 01:35:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721607#action_12721607

Namit Jain commented on HADOOP-6082:

The JobTracker needs a new API:

something like: 

decommissionTaskTrackers(int numberOfTT);

The TaskTracker may need to expose a new API to return its current load.

The JobTracker will get the current load of each TaskTracker and then decide to decommission
most lightly loaded 'n' takstrackers.

When a TaskTracker is being decommissioned, it will stop accepting new jobs, and will
die when all the current jobs are finished. This may lead to wastage of resources in the cluster.

The jobtracker can optionally pass a timeout after which the tasktracker will definitely die.
At that time, it might be a good idea to increase the number of retires for the tasks being

The UI may neeed to be changed to show the new status of the task tracker as well.

> Elegant decommission of lighty loaded tasktrackers from a map-reduce cluster
> ----------------------------------------------------------------------------
>                 Key: HADOOP-6082
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6082
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: dhruba borthakur
>            Assignee: Namit Jain
> There is a need to elegantly move some machines from one map-reduce cluster to another.
This JIRA is to discuss how to find lightly loaded tasktrackers that are candidates for decommissioning
and then to elegantly decommission them by waiting for existing tasks to finish.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message