hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit <lohit...@yahoo.com>
Subject Re: reduce more than one way
Date Fri, 07 Nov 2008 21:10:12 GMT
There is mapper called IdentityMapper (look of IdentityMapper.java), which basically reads
input and outputs without doing anything.
May be you can run your mapper with no reducers and store intermediate output and then run
your 2 hadoop programs with Identity mapper and different set of reducers.

----- Original Message ----
From: Elia Mazzawi <elia.mazzawi@casalemedia.com>
To: core-user@hadoop.apache.org
Sent: Friday, November 7, 2008 12:35:44 PM
Subject: reduce more than one way


I'm writing hadoop programs in Java,
I have 2 hadooop map/reduce programs that have the same map, but a different reduce methods.

can i run them in a way so that the map only happens once?

maybe store the intermediate result or something?

View raw message