Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of pramybhats@googlemail.com
 designates 74.125.82.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=iA0mdxEKCHiTykZvjWTsqSIyJ8p/IQDirLomMmxGdh2CKe8GOOKRrcLNt22boyLrGv
         EuXFqJdSJpo9PGfg/y3QUCvI75n9Dpexwi44+FgXYfqapFj5i2/NjfFCxRGyT/bKcWQF
         cGCCjSZp4H4bi1OhARQYrgp1qKHvqsPJ0Uisg=
MIME-Version: 1.0
In-Reply-To: <AANLkTilvvAIgs504igTN0Vveau8mPIljIa4FN8iGYfSO@mail.gmail.com>
References: <AANLkTinu8TgbidALiWi-NQBjshQENdU8bUSEJc54yQA1@mail.gmail.com>
	<AANLkTikZJk3hzwnbfUfzazFptP0-5DPNgAk-MsIo4Kq2@mail.gmail.com>
	<AANLkTilHOnjTpWN9nwyIMG_XF2CehicHeR59sKub5J5g@mail.gmail.com>
	<AANLkTilZRoHMflxdgiGfrCRtHCvmKyG_VOPwnlX1bqla@mail.gmail.com>
	<AANLkTilvvAIgs504igTN0Vveau8mPIljIa4FN8iGYfSO@mail.gmail.com>
Date: Fri, 2 Jul 2010 08:05:25 +0200
Message-ID: <AANLkTinXrkYCTqWVzWk3GdB--317S7vAaRXy7frJEPgY@mail.gmail.com>
Subject: Re: Intermediate files generated.
From: Pramy Bhats <pramybhats@googlemail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e657b25a7d4b8c048a615f93

--0016e657b25a7d4b8c048a615f93
Content-Type: text/plain; charset=ISO-8859-1

Hi Hemanth,

I need to use the output of the mapper for some other application. As a
result, if I can redirect the output of the map in temp files of my choice
(which are stored on hdfs) then i can reuse the output later. At the same
time, the succeeding reducer can read the input from this temp files without
any overhead.

thanks,
--PB

On Fri, Jul 2, 2010 at 3:52 AM, Hemanth Yamijala <yhemanth@gmail.com> wrote:

> Alex,
>
> > I don't think this is what I am looking for. Essential, I wish to run
> both
> > mapper as well as reducer. But at the same time, i wish to make sure that
> > the temp files that are used between mappers and reducers are of my
> choice.
> > Here, the choice means that I can specify the files in HDFS that can be
> used
> > as temp files.
>
> Could you explain why you want to do this ?
>
> >
> > thanks,
> > --PB.
> >
> > On Fri, Jul 2, 2010 at 12:14 AM, Alex Loddengaard <alex@cloudera.com>
> wrote:
> >
> >> You could use the HDFS API from within your mapper, and run with 0
> >> reducers.
> >>
> >> Alex
> >>
> >> On Thu, Jul 1, 2010 at 3:07 PM, Pramy Bhats <pramybhats@googlemail.com
> >> >wrote:
> >>
> >> > Hi,
> >> >
> >> > I am using hadoop framework for writing MapReduce jobs. I want  to
> >> redirect
> >> > the output of Map into files of my choice and later use those files as
> >> > input
> >> > for Reduce phase.
> >> >
> >> >
> >> > Could you please suggest, how to proceed for it ?
> >> >
> >> > thanks,
> >> > --PB.
> >> >
> >>
> >
>

--0016e657b25a7d4b8c048a615f93--