hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amareshwari Sri Ramadasu <amar...@yahoo-inc.com>
Subject Re: multiple outputs
Date Wed, 09 Jun 2010 04:42:20 GMT
Yes. They can be used inside a mapper also.
See org.apache.hadoop.mapred.lib.TestMultipleOutputs.java or org.apache.hadoop.mapreduce.lib.output.TestMRMultiplteOutputs
for some sample code.


On 6/9/10 5:57 AM, "Torsten Curdt" <tcurdt@vafer.org> wrote:

Can the MultipleOutputs also be used inside a mapper?

So basically I pipe data into different reducers from the mapper.

Of course I could do two separate jobs but that would very inefficient
as I would have to go/read through all the data twice.


On Tue, Jun 8, 2010 at 06:22, Amareshwari Sri Ramadasu
<amarsri@yahoo-inc.com> wrote:
> MultipleOutputs is ported to use new api through
> http://issues.apache.org/jira/browse/MAPREDUCE-370
> See the discussions on jira and javadoc/testcase as an example on how to use
> it.
> Thanks
> Amareshwari
> On 6/7/10 8:08 PM, "Torsten Curdt" <tcurdt@apache.org> wrote:
> I need to emit to different output files from a reducer.
> The old API had MultipleSequenceFileOutputFormat.
> Am I missing something or is this gone in the new API?
> Are there any problems porting this over?
> Or does it just needs to be done?
> cheers
> --
> Torsten

View raw message