hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chang Hu" <chan...@cs.umd.edu>
Subject Re: setMapOutputValueClass doesn't work
Date Mon, 24 Mar 2008 21:06:35 GMT
Thanks Riccardo,  but that's not the case.  I checked and made sure it's
collecting FooWritable.  In fact, from the following thread:

http://www.nabble.com/Different-output-classes-from-map-and-reducer-td15728122.html

My exception is the same as if map output value class was not set.


 - Chang

On Mon, Mar 24, 2008 at 4:52 PM, Riccardo Boscolo <drboscolo@netseer.com>
wrote:

> From the exception stack it appears that the map output class is correctly
> set to FooWritable.class but you are trying to collect BarWritable(s) in
> your map tasks.
>
> Best,
>
> RB
>
> On Mon, Mar 24, 2008 at 1:22 PM, Chang Hu <changhu@cs.umd.edu> wrote:
>
> > Hi,
> >
> > I have been unsuccessfully trying to set the map output value class
> > different to the one reduce outputs (in 0.16.0).  AFAIK the following
> > should
> > do the trick:
> >
> > conf.setMapOutputValueClass(FooWritable.class)
> > conf.setOutputValueClass(BarWritable.class)
> >
> > However I kept getting exceptions saying BarWritable is not FooWritable.
> > Both classes implements WritableComparable.
> >
> > java.io.IOException: wrong value class: test.BarWritable is not class
> > test.FooWritable
> >    at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java
> > :952)
> >    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$2.collect(
> > MapTask.java:489)
> >    at edu.umd.cs.hadoopImage.HadoopImg$InpaintReduceClass.reduce(
> > HadoopImg.java:122)
> >    at edu.umd.cs.hadoopImage.HadoopImg$InpaintReduceClass.reduce(
> > HadoopImg.java:1)
> >    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(
> > MapTask.java:522)
> >    at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(
> > MapTask.java:493)
> >    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(
> MapTask.java
> > :713)
> >    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:209)
> >    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java
> > :2071)
> >
> >
> > Suggestions?  Thanks.
> >
> >
> >  - Chang
> >
> >
> > --
> > ---------------
> > √úberstehen ist alles.
> >
> >
> > Chang Hu
> > Ph.D. student
> > Computer Science Department
> > University of Maryland
> >
>
>
>
> --
> -------------------------------
> Riccardo Boscolo, PhD
> V.P. of Core Technology
> Netseer Inc.
> 11943 Montana Ave, Suite 200
> Los Angeles, CA 90049
> T: 310-597-4482
> F: 310-597-4489
> Email: drboscolo@netseer.com
> -------------------------------
>



-- 
---------------
√úberstehen ist alles.


Chang Hu
Ph.D. student
Computer Science Department
University of Maryland

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message