Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of wenhao.xu@gmail.com
 designates 209.85.214.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=xNX1rLf7A/7e4iZSttbtxkhHw2j0JQtdOcwSYH3/5xLw/q2psTBOueP+WRdbGyFUz4
         rqLEszj/MuoLUnajDPNObS/FIfS0EkbuWcUt1QR4gchiVJ2/7g09jwYado+V/ZbCmH+c
         k6VJb3MI3nG/mQtMHeysom4cxdtCf90fA2jtY=
MIME-Version: 1.0
Date: Wed, 16 Feb 2011 19:02:04 +0800
Message-ID: <AANLkTinVHbfoxUfC_PMF+N_bYs=8tLRG4E9XTk-bYBD3@mail.gmail.com>
Subject: Could we use different output Format for the Mapper and Combiner?
From: Stanley Xu <wenhao.xu@gmail.com>
To: mapreduce-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=20cf304346c60e5f1b049c6436ed

--20cf304346c60e5f1b049c6436ed
Content-Type: text/plain; charset=ISO-8859-1

Dear all,

I am writing a map-reduce job today. Which I hope I could use different
format for the Mapper and Combiner. I am using the Text as the format of the
Mapper and MapWritable as the format of the format.

But it looks the hadoop didn't support that yet?

I have some code like the following:

public class RawLogMapper extends Mapper<LongWritable, Text, Text, Text> {

public class RawLogCombiner extends Reducer<Text, Text, Text, MapWritable> {

job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(MapWritable.class);
job.setOutputFormatClass(TextOutputFormat.class);

But it failed and the logs told me that there are type mismatch. Is there
anyway I could use different type for the VALUEOUT for the mapper and
combiner?

Thanks


Best wishes,
Xu Wenhao

--20cf304346c60e5f1b049c6436ed
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Dear all,<div><br></div><div>I am writing a map-reduce job today. Which I h=
ope I could use different format for the Mapper and Combiner. I am using th=
e Text as the format of the Mapper and MapWritable as the format of the for=
mat.</div>
<div><br></div><div>But it looks the hadoop didn&#39;t support that yet?</d=
iv><div><br></div><div>I have some code like the following:</div><div><br><=
/div><div><div>public class RawLogMapper extends Mapper&lt;LongWritable, Te=
xt, Text, Text&gt; {</div>
<div><br></div><div><div>public class RawLogCombiner extends Reducer&lt;Tex=
t, Text, Text, MapWritable&gt; {</div></div><div><div><br></div><div>job.se=
tMapOutputKeyClass(Text.class);</div><div>job.setMapOutputValueClass(Text.c=
lass);</div>
<div><br></div><div>job.setOutputKeyClass(Text.class);</div><div>job.setOut=
putValueClass(MapWritable.class);</div><div>job.setOutputFormatClass(TextOu=
tputFormat.class);</div></div><div><br></div><div>But it failed and the log=
s told me that there are type mismatch. Is there anyway I could use differe=
nt type for the VALUEOUT for the mapper and combiner?</div>
<div><br></div><div>Thanks</div><div><div><br></div><div><br clear=3D"all">=
Best wishes,<br>Xu Wenhao
</div></div></div>

--20cf304346c60e5f1b049c6436ed--