hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron <aaron_huntnf...@yahoo.com>
Subject Re: type mismatch, newbie help
Date Fri, 16 Oct 2009 03:45:54 GMT
Yep that was it. I changed the method signature to LongWriteable for  
the 1st parameter (also annotated it w/ Override to see the compile  
error)

thanks for your help!


On Oct 15, 2009, at 6:40 PM, Ahad Rana wrote:

> Hi,
> Unfortunately Mapper is now a class and from your call stack it  
> seems that
> Mapper's default  map implementation is being called (instead of the  
> one you
> defined in your class), which is passing the LongWritable key to the
> collector. You should use @Override to have the compiler help you  
> figure out
> why your map function doesn't have the same signature as the one  
> defined in
> the base class.
>
> Best of luck.
>
> Ahad.
>
> On Thu, Oct 15, 2009 at 10:29 AM, yz5od2 <woods5242- 
> outdoors@yahoo.com>wrote:
>
>> Hi,
>> I am new to Hadoop so this might be an easy question for someone to  
>> help me
>> with.
>>
>> I continually am getting this exception (my code follows below)
>>
>> java.io.IOException: Type mismatch in key from map: expected
>> org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable
>>       at
>> org.apache.hadoop.mapred.MapTask 
>> $MapOutputBuffer.collect(MapTask.java:807)
>>       at
>> org.apache.hadoop.mapred.MapTask 
>> $NewOutputCollector.write(MapTask.java:504)
>>       at
>> org 
>> .apache 
>> .hadoop 
>> .mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java: 
>> 80)
>>       at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
>>       at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java: 
>> 583)
>>       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>       at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>
>>
>> Running 0.20.1
>>
>> I have a text file with lines of data separated by carriage  
>> returns. This
>> is properly stored in a directory within HDFS. I only have a  
>> Mapping task
>> for processing this file, after the mapping is done it should go  
>> straight to
>> output, No reduce or combiner functions.
>>
>> I am just trying to test to see if this will run. The mapper just  
>> takes the
>> data sent to it and adds it back to the collector as 2 text values.
>>
>> --------------------
>> My Mapper:
>> --------------------
>>
>> public class MyTestMapper extends  
>> Mapper<LongWritable,Text,Text,Text> {
>>
>>       public void map(Object key, Text value, Context context) throws
>> IOException, InterruptedException {
>>               String key = key.toString();
>>               String line = value.toString();
>>
>>               String id = extractId(line);
>>               String reformattedLine = reformatLine(line);
>>
>>               context.write(new Text(id), new Text(reformattedLine));
>>       }
>> }
>>
>> --------------------
>> My job submission code:
>> -------------------------------------
>>
>> Job job = new Job(conf);
>> job.setJarByClass(MyTestMapper.class);
>> job.setMapperClass(MyTestMapper.class);
>>
>> FileInputFormat.addInputPath(job, new Path("/myDir/sample.txt"));
>> FileOutputFormat.setOutputPath(job, new
>> Path("/myDir/output/results-"+System.currentTimeMillis()+".txt"));
>>
>> job.setMapOutputKeyClass(Text.class);
>> job.setMapOutputValueClass(Text.class);
>>
>> job.setOutputKeyClass(Text.class);
>> job.setOutputValueClass(Text.class);
>>
>> job.submit();



Mime
View raw message