Hello,
On Fri, Oct 29, 2010 at 12:45 PM, Jeff Zhang <zjffdu@gmail.com> wrote:
> TaskTracker will tell JobTracker how many free slots it has through
> heartbeat. And JobTracker will choose the best tasktracker with the
> consideration of data locality.
Yes. To add some more, a scheduler is responsible to do assignments of
tasks (based on various stats, including data locality) to proper
tasktrackers. Scheduler.assignTasks(TaskTracker) is used to assign a
TaskTracker its tasks, and the scheduler type is configurable (Some
examples are Eager/FIFO scheduler, Capacity scheduler, etc.).
This scheduling is done when a heart beat response is to be sent back
to a TaskTracker that called JobTracker.heartbeat(...).
>
>
> On Thu, Oct 28, 2010 at 2:52 PM, Zhenhua Guo <jenvor@gmail.com> wrote:
>> Hi, all
>> I wonder how Hadoop schedules mappers and reducers (e.g. consider
>> load balancing, affinity to data?). For example, how to decide on
>> which nodes mappers and reducers are to be executed and when.
>> Thanks!
>>
>> Gerald
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
Harsh J
www.harshj.com
|