hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramy Bhats <pramybh...@googlemail.com>
Subject Re: Intermediate files generated.
Date Thu, 01 Jul 2010 23:08:03 GMT
Hi Alex,

I don't think this is what I am looking for. Essential, I wish to run both
mapper as well as reducer. But at the same time, i wish to make sure that
the temp files that are used between mappers and reducers are of my choice.
Here, the choice means that I can specify the files in HDFS that can be used
as temp files.

thanks,
--PB.

On Fri, Jul 2, 2010 at 12:14 AM, Alex Loddengaard <alex@cloudera.com> wrote:

> You could use the HDFS API from within your mapper, and run with 0
> reducers.
>
> Alex
>
> On Thu, Jul 1, 2010 at 3:07 PM, Pramy Bhats <pramybhats@googlemail.com
> >wrote:
>
> > Hi,
> >
> > I am using hadoop framework for writing MapReduce jobs. I want  to
> redirect
> > the output of Map into files of my choice and later use those files as
> > input
> > for Reduce phase.
> >
> >
> > Could you please suggest, how to proceed for it ?
> >
> > thanks,
> > --PB.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message