hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: Assign different task to the workers.
Date Tue, 06 Oct 2009 22:06:25 GMT
The same mapper class will be used for all map tasks. Individual tasks could
perhaps do something clever like recover their task ids and do something
different based on task id modulo the number of distinct task types, but in
general, the idea behind MapReduce is that all tasks should be applying the
same process to all of your data.

A single MapReduce job is specifically not intended to be utilized as a
pipeline of different types of workers. For that, you should use multiple
MapReduce jobs in series.

- Aaron

2009/10/6 María Helena Mejía Salazar <mhmejia@gmail.com>

> Hi,
> The same task can be done  in different ways,  i.e a sort task can be done
> by merge sort, heap sort etc.
> Is possible in  MapReduce assign different task to the workers?  Can the
> master assign a merge sort task to one worker and assign heap sort task to
> another worker?
> Thanks,
> Maria

View raw message