hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamesh <kames...@imaginea.com>
Subject Re: Can we use Inheritance hierarchy to specify the outputvalue class for mapper which is also inputvalue class for the reducer ?
Date Wed, 05 Oct 2011 09:38:39 GMT
On Tuesday 04 October 2011 10:16 PM, Anuja Kulkarni wrote:
> Hi,
>
> We have class hierarchy for output value for both mapper as well as 
> reducer class   as parent (abstract class) , child1,child2,…
> We have mapper class which is specified with its outputvalue class as 
> parent class ; the map function will emit  either  child1 or child2 
> depending on the logic we used ( and reducer is having its inputvalue 
> class as parent class).
> But we are getting error as "*java.io.IOException: Type mismatch in 
> value from map: expected **parent classreceived child class”*
> So is this possible to specify parent class as output/inputvalue class 
> in case of mapper as well as in reducer ; since we want to follow some 
> object-oriented approach. What will be the correct way of achieving this ?
>
>
> - Anuja
Hi Anuja,

bq. So is this possible to specify parent class as output/inputvalue 
class in case of mapper as well as in reducer ;

I **think** it is not possible.

bq. What will be the correct way of achieving this ?

I feel the following approach works (there could be a better way of 
doing this)

Create a TupleWritable instance with either child1 or child2 array as an 
argument based on your logic at the mapper side and write the same 
TupleWritable instance as a value of Mapper output value.

At the reducer side, get the writable type from the TupleWritable and do 
instanceof check to determine the appropriate value type.

-- 
Thanks&Regards,
Bh.V.S.Kamesh.


Mime
View raw message