hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmad Humayun <ahmad.hu...@gmail.com>
Subject Re: why reduce task can be scheduled before map tasks are 100% completed?
Date Sun, 02 Aug 2009 12:18:13 GMT
Even though reduces are scheduled, they just transfer data from maps before
all maps are completed.The reduce only begins its processing when all maps
are completed.

Please correct me if I'm wrong.


2009/8/2 我的Gmail邮箱 <zhangxiang390@gmail.com>

> Hi, everyone.
> In class org.apache.hadoop.mapred.JobInProgress, there is a public method:
> scheduleReduces(), it will return true if "finishedMapTasks >=
> completedMapsForReduceSlowstart"
> and then the scheduler can schedule a new reduce task for a given
> taskTracker.
> but as I konw, reduce can not be started unitl map is 100% completed. Does
> anyone can explain it? thanks a lot.

Ahmad Humayun
Research Associate
Computer Science Dpt., LUMS
+92 321 4457315

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message