Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.210.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAMe0HzQdxqeoUjsox6OheC2FcZ7XVc0syEWzOwF-6aTHEh0feQ@mail.gmail.com>
References: 
 <CAMe0HzRPB7EJwFtLw8RLEp1-1Yd6ZtKHgMBRvpLoT6r8zUpWZA@mail.gmail.com>
 <CAOcnVr238199RUoMSy9ymXqGTfXAN6EGTM_RcL4cgmG8gx7BMQ@mail.gmail.com>
 <CAMe0HzQdxqeoUjsox6OheC2FcZ7XVc0syEWzOwF-6aTHEh0feQ@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Sun, 24 Mar 2013 19:46:30 +0530
Message-ID: 
 <CAOcnVr3ygJ=qcsbVns+PWbA7q7oBh993=2B4cZ32FQb4HOQy1A@mail.gmail.com>
Subject: Re: 2 Reduce method in one Job
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Yes, just use an identity mapper (in new API, the base Mapper class
itself identity-maps, in the old API use IdentityMapper class) and set
the input path as the output path of the first job.

If you'll be ending up doing more such step-wise job chaining,
consider using Apache Oozie's workflow system.

On Sun, Mar 24, 2013 at 7:23 PM, Fatih Haltas <fatih.haltas@nyu.edu> wrote:
> Thank you very much.
>
> You are right Harsh, it is exactly what i am trying to do.
>
> I want to process my result, according to the keys and i donot spend time
> writing this data to hdfs, I want to pass data as input to another reduce.
>
> One more question then,
> Creating 2 diffirent job, secondone has only reduce for example, is it
> possible to pass first jobs output as argument to second job?
>
>
> On Sun, Mar 24, 2013 at 5:44 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> You seem to want to re-sort/partition your data without materializing
>> it onto HDFS.
>>
>> Azuryy is right: There isn't a way right now and a second job (with an
>> identity mapper) is necessary. With YARN this is more possible to
>> implement into the project, though.
>>
>> The newly inducted incubator project Tez sorta targets this. Its in
>> its nascent stages though (for general user use), and the website
>> should hopefully appear at
>> http://incubator.apache.org/projects/tez.html soon. Meanwhile, you can
>> read the proposal behind this project at
>> http://wiki.apache.org/incubator/TezProposal. Initial sources are at
>> https://svn.apache.org/repos/asf/incubator/tez/trunk/.
>>
>> On Sun, Mar 24, 2013 at 6:33 PM, Fatih Haltas <fatih.haltas@nyu.edu>
>> wrote:
>> > I want to get reduce output as key and value then I want to pass them to
>> > a
>> > new reduce as input key and input value.
>> >
>> > So is there any Map-Reduce-Reduce kind of method?
>> >
>> > Thanks to all.
>>
>>
>>
>> --
>> Harsh J
>
>


--
Harsh J