Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of chenyangyinpeng@gmail.com
 designates 209.85.132.240 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:references;
        b=JiRNjabbHLj83u6uhsXJ3xJaPW153hklprtOEQNvKbJ7Jd2fqD0rG+L3f0a8Zh7wj7
         tMETbS/zHcv2bd6Uam14d94yzo62yeq+lEt92G74Yfl1Py3v/ST3XyaxAc7Tz0KSVJ5J
         d+ycvPHfvJJ+Knmf1Mrf3z1Sd+nEnX5hQ38jI=
Message-ID: <dc8cbe0e0806051714m2139b7b4yf6a9d1a55252d5ca@mail.gmail.com>
Date: Thu, 5 Jun 2008 20:14:07 -0400
From: "Yang Chen" <chenyangyinpeng@gmail.com>
To: core-user@hadoop.apache.org
Subject: Re: MapWritable as output value of Reducer
In-Reply-To: <e75c02ef0806051636i3fc67813rc98d5f49e2ad0d8f@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_8534_26723564.1212711247646"
References: <e75c02ef0806051636i3fc67813rc98d5f49e2ad0d8f@mail.gmail.com>

------=_Part_8534_26723564.1212711247646
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

I believe the (key, value) structure is same both input and output file. In
this case, you can consider the job flow.
Like below,
      JobConf confA = new JobConf(A.class);
     conf.setJobName("A");
     conf.setOutputKeyClass(Text.class);
     conf.setOutputValueClass(IntWritable.class);
     conf.setMapperClass(MapA.class);
     conf.setCombinerClass(ReduceA.class);
     conf.setReducerClass(ReduceA.class);
     conf.setInputFormat(TextInputFormat.class);
     conf.setOutputFormat(TextOutputFormat.class);
     conf.setInputPath(new Path("/inputA"));
     conf.setOutputPath(new Path("/outputA"));
     JobClient.runJob(confA);

      JobConf confB = new JobConf(B.class);
     conf.setJobName("B");
     conf.setOutputKeyClass(Text.class);
     conf.setOutputValueClass(IntWritable.class);
     conf.setMapperClass(MapB.class);
     conf.setCombinerClass(ReduceB.class);
     conf.setReducerClass(ReduceB.class);
     conf.setInputFormat(TextInputFormat.class);
     conf.setOutputFormat(TextOutputFormat.class);
     conf.setInputPath(new Path("/outputA"));
     conf.setOutputPath(new Path("/outputB"));
     JobClient.runJob(confB);

On Thu, Jun 5, 2008 at 7:36 PM, Tarandeep Singh <tarandeep@gmail.com> wrote:

> hi,
>
> Can I use MapWritable as an output value of a Reducer ?
>
> If yes, how will the (key, value) pairs in the MapWritable object will be
> written to the file ? What output format should I use in this case ?
>
> Further, I want to chain the output of the first map reduce job to another
> map reduce job, so in the second map reduce job, what input format should I
> specify ?
>
> Can I reconstruct the MapWritable objects in the mapper of the second job ?
>
> Thanks,
> Taran
>

------=_Part_8534_26723564.1212711247646--