hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: what is the difference between mapper and identity mapper, reducer and identity reducer?
Date Thu, 05 Sep 2013 14:01:22 GMT
Identity Mapper and Reducer just like the concept of Identity function in
mathematics i.e. do not transform the input and return it as it is in
output form. Identity Mapper takes the input key/value pair and spits it
out without any processing.

The case of identity reducer is a bit different. It does not mean that the
reduce step will not take place. It will take place and the related sorting
and shuffling will also be performed but there will be no aggregation. So
you can use identity reducer if you want to sort your data that is coming
from map but don't care for any grouping and also fine with multiple
reducer outputs (unlike using 1 reducer.)

Regards,
Shahab



On Thu, Sep 5, 2013 at 9:43 AM, mallik arjun <mallik.cloud@gmail.com> wrote:

> hi  all,
>
> please  tell me what is the difference between mapper and identtiy mapper
> , reducer and identity reducer.
>
> thanks in advance.
>

Mime
View raw message