hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Sammer <esam...@cloudera.com>
Subject Re: number of reducers
Date Sun, 06 Jun 2010 23:31:20 GMT
Torsten:

To clarify, how many reducers do you actually see? (i.e. Do you see 4
reducers or 1?) It should work as you expect.

On Sun, Jun 6, 2010 at 1:33 PM, Torsten Curdt <tcurdt@vafer.org> wrote:
> When I set
>
>  job.setPartitionerClass(MyPartitioner.class);
>  job.setNumReduceTasks(4);
>
> I would expect to see my MyParitioner get called with
>
>  getPartition(key, value, 4)
>
> but still I see it only get called with 1.
>
> If also tried setting
>
>                conf.set("mapred.map.tasks.speculative.execution", "false");
>                conf.set("mapred.reduce.tasks.speculative.execution", "false");
>
> What am I missing here?
>
> cheers
> --
> Torsten
>



-- 
Eric Sammer
phone: +1-917-287-2675
twitter: esammer
data: www.cloudera.com

Mime
View raw message