hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: how can I get the number of reducer in Map
Date Tue, 26 Jul 2011 09:19:34 GMT
Ah wait, guess I figured your problem -- you may not be reutilizing
the Configuration instance inside your mapper.

Override the configure() method in your mapper and get the value out
of the configuration instance passed to the mapper instead of
instantiating a new one (with defaults).

2011/7/26 rabbit_cheng <rabbit_cheng@126.com>:
> In my map function, I  need to know the number of reducer, the code segment
> in my program like this:
>    JobConf job = new JobConf(driverClass.class);
>     int numReducer=job.getNumReduceTasks();
> but the function invocation job.getNumReduceTasks() always returns the value
> of "1". I have tested many times: In driver function, I set the number of
> reducer to 2 using the following statements:
>     JobConf job =new JobConf(driverClass.class);
>     job.setNumReduceTasks(2);
> As a result, the reducers' output file number is 2, so I know the
> statements above worked, but what I get in map function is still to be "1".
> How can I get the right number of reducer in map function? thanks!

Harsh J

View raw message