hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <...@yahoo-inc.com>
Subject Re: About the support of MapReduce task scheduling schemes
Date Wed, 13 Feb 2008 17:53:58 GMT

On Feb 13, 2008, at 9:39 AM, Jaideep Dhok wrote:

> Hi,
> AFAIK right now they are only doing FCFS scheduling. You can read  
> the code
> in org.apache.hadoop.mapred.JobTracker.java. I think the code in
> "getNewTaskForTaskTracker" method.

It actually prefers a host where the data is local. On large jobs, we  
get 95% of the maps running locally (assuming hdfs cluster == map/ 
reduce cluster). You also adding rack locality into the scheduling in  
HADOOP-1985.

-- Owen

Mime
View raw message