hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: [VOTE] Direction for Hadoop development
Date Wed, 08 Dec 2010 18:55:58 GMT
On 12/07/2010 10:26 AM, Konstantin Shvachko wrote:
>> I no longer think we should add any new serialization implementations to
> the kernel.
> Not clear. Do you propose to keep current serialization(s) and not add new
> ones?
> Or do you propose to replace current serialization by abstract interfaces
> and move implementations to libraries?

We can't move existing serialization implementations to an optional 
library without breaking compatibility.  Long-term that might be nice, 
but I am not proposing that short-term.  Short-term I propose we avoid 
adding new serialization implementations to the default classpath, 
especially those that add new dependencies to every task.  Long-term we 
might split library code into perhaps a few categories:
  - mandatory: this might include, e.g., IdentityMapper and 
IdentityReducer, the default implementations.
  - back-compatible: the collection of library components that were 
provided on the default classpath and can be enabled for back-compatible 
  - optional: components that jobs can optionally depend on.  This is 
where new components that are not mandatory would be added.


View raw message