hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: Does the class of the Mapper output need to match the exact class of the specified output?
Date Tue, 02 Feb 2010 16:18:20 GMT

On Jan 26, 2010, at 5:17 PM, Wilkes, Chris wrote:

> Does this level of checking really need to be done?

For Writables, the simple answer is yes. It happens because the way  
you deserialize is to:

Writable obj = ReflectionUtils.newInstance(cls, conf);
obj.readFields(in);

So you need to know the specific class to read into. To support this  
use case, there is library code available in Hadoop:

http://bit.ly/9UoZGc

There is work underway to loosen the restriction in trunk, but that is  
a long ways from being released.

-- Owen
Mime
View raw message