hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Costa <psdc1...@gmail.com>
Subject Re: PiEstimator error - Type mismatch in key from map
Date Thu, 27 Jan 2011 10:19:36 GMT
Thanks Nicholas, but it didn't worked.

Can I do a remote debugging on hadoop examples? I really like to put a
breakpoint in the Pi class.

Thanks,

On Wed, Jan 26, 2011 at 6:46 PM, Tsz Wo (Nicholas), Sze
<s29752-hadoopuser@yahoo.com> wrote:
> Okay, I got it now.  You were talking about your programs but not the
> PiEstimator example came from Hadoop.  Then, you have to set
> "mapred.output.key.class" and "mapred.output.value.class" as Srihari
> mentioned.  Below are the APIs.
>
>     //new API
>     final Job job = ...
>     job.setMapOutputKeyClass(BooleanWritable.class);
>     job.setMapOutputValueClass(LongWritable.class);
>
>     //old API
>     final JobConf jobconf = ...
>     jobconf.setOutputKeyClass(BooleanWritable.class);
>     jobconf.setOutputValueClass(LongWritable.class);
>
> Nicholas
>
> ________________________________
> From: Srihari Anantha Padmanabhan <sriharia@yahoo-inc.com>
> To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
> Sent: Wed, January 26, 2011 10:36:09 AM
> Subject: Re: PiEstimator error - Type mismatch in key from map
>
> I am using Hadoop 0.20.2. I just wrote my own map-reduce program based on
> the map-reduce tutorial at
> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html
>
> On Jan 26, 2011, at 10:27 AM, Pedro Costa wrote:
>
>> Hadoop 20.1
>>
>> On Wed, Jan 26, 2011 at 6:26 PM, Tsz Wo (Nicholas), Sze
>> <s29752-hadoopuser@yahoo.com> wrote:
>>> Hi Srihari,
>>>
>>> Same questions to you: Which version of Hadoop are you using?  And where
>>> did
>>> you get the examples?  I guess you were able to reproduce it.  I suspect
>>> the
>>> examples and the Hadoop are in different versions.
>>>
>>> Nicholas
>>>
>>>
>>> ________________________________
>>> From: Srihari Anantha Padmanabhan <sriharia@yahoo-inc.com>
>>> To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
>>> Sent: Wed, January 26, 2011 10:15:08 AM
>>> Subject: Re: PiEstimator error - Type mismatch in key from map
>>>
>>> I got a similar error before in one of my projects. I had to set the
>>> values
>>> for "mapred.output.key.class" and "mapred.output.value.class".
>>> That resolved the issue for me.
>>> Srihari
>>> On Jan 26, 2011, at 10:09 AM, Pedro Costa wrote:
>>>
>>> Yes, I can reproduce it deterministically. But, I also did some
>>> changes to the Hadoop MR code. Most definitely this is the reason. I'm
>>> looking throughly through the code.
>>>
>>> I'll say something after I find the problem.
>>>
>>> I was just wondering if this error has happened to someone before.
>>> Maybe I could get a hint and try to see what's my problem easily.
>>>
>>> Thanks,
>>>
>>> On Wed, Jan 26, 2011 at 6:02 PM, Tsz Wo (Nicholas), Sze
>>> <s29752-hadoopuser@yahoo.com> wrote:
>>>
>>> Hi Pedro,
>>>
>>> This is interesting.  Which version of Hadoop are you using?  And where
>>> did
>>>
>>> you get the example class files?  Also, are you able to reproduce it
>>>
>>> deterministically?
>>>
>>> Nicholas
>>>
>>> ________________________________
>>>
>>> From: Pedro Costa <psdc1978@gmail.com>
>>>
>>> To: mapreduce-user@hadoop.apache.org
>>>
>>> Sent: Wed, January 26, 2011 5:47:01 AM
>>>
>>> Subject: PiEstimator error - Type mismatch in key from map
>>>
>>> Hi,
>>>
>>> I run the PI example of hadoop, and I've got the following error:
>>>
>>> [code]
>>>
>>> java.io.IOException: Type mismatch in key from map: expected
>>>
>>> org.apache.hadoop.io.BooleanWritable, recieved
>>>
>>> org.apache.hadoop.io.LongWritable
>>>
>>>    at
>>>
>>>
>>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:885)
>>>
>>>    at
>>>
>>>
>>> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:551)
>>>
>>>    at
>>>
>>>
>>> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:81)
>>>
>>>    at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
>>>
>>>    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>>
>>>    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:637)
>>>
>>>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>>>
>>>    at org.apache.hadoop.mapred.Child.main(Child.java:190)
>>>
>>> [/code]
>>>
>>> I've look at the map function of the class "PiEstimator.class" and it
>>> seems
>>>
>>> ok.
>>>
>>> [code]
>>>
>>> public void map(LongWritable offset,
>>>
>>>        LongWritable size,
>>>
>>>        OutputCollector<BooleanWritable, LongWritable> out,
>>>
>>>        Reporter reporter) throws IOException {}
>>>
>>> [/code]
>>>
>>>
>>> What's wrong with this examples?
>>>
>>> Thanks,
>>>
>>> --
>>>
>>> Pedro
>>>
>>>
>>>
>>>
>>> --
>>> Pedro
>>>
>>>
>>
>>
>>
>> --
>> Pedro
>
>



-- 
Pedro

Mime
View raw message