hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhou, Yunqing" <azure...@gmail.com>
Subject Re: Can a MapReduce task only consist of a Map step?
Date Mon, 21 Jul 2008 10:24:31 GMT
since the whole data is 5TB.  the Identity reducer still cost a lot of time.

On Mon, Jul 21, 2008 at 5:09 PM, Christian Ulrik S√łttrup <soettrup@nbi.dk>
wrote:

> Hi,
>
> you can simply use the built in reducer that just copies the map output:
>
> conf.setReducerClass(org.apache.hadoop.mapred.lib.IdentityReducer.class);
>
> Cheers,
> Christian
>
>
> Zhou, Yunqing wrote:
>
>> I only use it to do something in parallel,but the reduce step will cost me
>> additional several days, is it possible to make hadoop do not use a reduce
>> step?
>>
>> Thanks
>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message