hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: [jira] Commented: (HADOOP-115) Hadoop should allow the user to use SequentialFileOutputformat as the output format and to choose key/value classes that are different from those for map output.
Date Mon, 03 Apr 2006 17:45:16 GMT
Eric Baldeschwieler wrote:
> An observation...  this whole thread is about limits caused by type  
> safety.  Interestingly, the other implementation of map-reduce does  not 
> support types at all.  Everything is a string.
> 
> So I agree that our departure from the paper is the problem.  ;-)

A corollary is that one could simply use BytesWritable for all one's 
keys and values, altering only one's WritableComparator implementation, 
and one would not encounter this problem.  The use of types in Hadoop is 
thus an optional feature.  One could even layer a different type system 
on top of BytesWritable that exhibits the desired properties.

> I'm comfortable letting this lie for a while.  But I predict we've  not 
> heard the last of it.

Owen seems to be picking it up, which is fine by me.

Doug

Mime
View raw message