hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Teng, James" <xt...@ebay.com>
Subject FW: type mismatch error
Date Tue, 12 Jul 2011 06:45:46 GMT

hi, all.
I am a new hadoop beginner, I try to construct a map and reduce task to run, however encountered
an exception while continue going further.
Exception:
java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved
org.apache.hadoop.io.LongWritable
     at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
     at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
     at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
     at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Mapper:
public class TaskMapper extends Mapper<LongWritable , Text ,Text ,Text>{
      public void Map(LongWritable key, Text value, Context context)
      {
            try {
                  System.out.println(value.toString());
                  context.write(new Text(Calendar.getInstance().getTime().toString()),value);
            } catch (IOException e) {
                  // TODO Auto-generated catch block
                  e.printStackTrace();
            } catch (InterruptedException e) {
                  // TODO Auto-generated catch block
                  e.printStackTrace();
            }
      }
}
Reducer:
public class TaskReducer extends Reducer<Text , Text ,Text ,Text>{
      public void reduce(Text key,Iterable<Text> value, Context context)
      {
            try {
                  context.write(key,new Text(value.iterator().hashCode()+""));
            } catch (IOException e) {
                  // TODO Auto-generated catch block
                  e.printStackTrace();
            } catch (InterruptedException e) {
                  // TODO Auto-generated catch block
                  e.printStackTrace();
            }
      }

}
Main(Entry)
public class TaskEntry extends Configured implements Tool{

      @Override
      public int run(String[] arg0) throws Exception {
            Job job=new Job(getConf());
            job.setMapOutputKeyClass(Text.class);
            job.setMapOutputValueClass(Text.class);
            job.setOutputKeyClass(Text.class);
            job.setOutputValueClass(Text.class);
            //job.setInputFormatClass(KeyValueTextInputFormat.class);
            FileInputFormat.addInputPath(job, new Path(arg0[0]));
            FileOutputFormat.setOutputPath(job, new Path(arg0[1]));
            job.setMapperClass(TaskMapper.class);
            job.setReducerClass(TaskReducer.class);
            return job.waitForCompletion(true)?0:1;
      }
      public static void main(String args[]) throws Exception
      {
            int exitCode=ToolRunner.run(new TaskEntry(), args);
            System.exit(exitCode);
      }

}


Any help is greatly appreciated!
Thanks in advance.

James, Teng (Teng Linxiao)
eRL,   CDC,    eBay,    Shanghai
Extension:        86-21-28913530
MSN:     tenglinxiao@hotmail.com<mailto:tenglinxiao@hotmail.com>
Skype:                James,Teng
Email:            xteng@ebay.com<mailto:xteng@ebay.com>
[cid:image002.gif@01CC3FBD.676D31C0]

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message