hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rabbit_cheng <rabbit_ch...@126.com>
Subject Re:Re: how can I get the number of reducer in Map
Date Tue, 26 Jul 2011 09:36:58 GMT
yes, it works for me! I sincerely appreciate your help! thanks!

At 2011-07-26 17:19:34,"Harsh J" <harsh@cloudera.com> wrote:
>Ah wait, guess I figured your problem -- you may not be reutilizing
>the Configuration instance inside your mapper.
>Override the configure() method in your mapper and get the value out
>of the configuration instance passed to the mapper instead of
>instantiating a new one (with defaults).
>2011/7/26 rabbit_cheng <rabbit_cheng@126.com>:
>> In my map function, I  need to know the number of reducer, the code segment
>> in my program like this:
>>    JobConf job = new JobConf(driverClass.class);
>>     int numReducer=job.getNumReduceTasks();
>> but the function invocation job.getNumReduceTasks() always returns the value
>> of "1". I have tested many times: In driver function, I set the number of
>> reducer to 2 using the following statements:
>>     JobConf job =new JobConf(driverClass.class);
>>     job.setNumReduceTasks(2);
>> As a result, the reducers' output file number is 2, so I know the
>> statements above worked, but what I get in map function is still to be "1".
>> How can I get the right number of reducer in map function? thanks!
>Harsh J

View raw message