hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <ja...@attributor.com>
Subject Re: how to use two reduce fucntions?
Date Sat, 23 Feb 2008 19:29:31 GMT
If you set up a partitioner class, you could pre partition the output of 
the into the relevant segments.
Then your reducer would be responsible for determining which reduce 
function to apply based on which segment the key is part of.

Amar Kamat wrote:
> Can you provide more details on what exactly what you wish to do? What 
> is the nature of reducers? A simple answer would be with map(m) and 
> reducers(r1,r2) you can run 2 jobs i.e job1(m,r1) and 
> job2(IdentityMapper,r2). But it depends what exactly r1 and r2 do. 
> Also combiners will play an important role. Also can one merge r1 and 
> r2 to r and run a job(m,r)
> Amar
> On Sat, 23 Feb 2008, ma qiang wrote:
>> Hi all,
>>    I have a program need to use two reduce fucntions, who can tell me 
>> why?
>>    Thank you!
>> Qiang

Jason Venner
Attributor - Publish with Confidence <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers, contact if interested

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message