hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@yahoo-inc.com>
Subject Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?
Date Sat, 06 Sep 2008 00:21:55 GMT

On Aug 15, 2008, at 10:03 PM, Nigel Daley wrote:

>> Another benefit is that it would increase the separation of these  
>> technologies, so that, e.g., folks could more easily run different  
>> versions of mapreduce on top of different versions of HDFS.   
>> Currently we make no such guarantees.  Folks would be able to  
>> upgrade to, e.g., the next release of mapreduce on a subset of  
>> their cluster without upgrading their HDFS.  That's not currently  
>> supported.  As we move towards splitting mapreduce into a scheduler  
>> and runtime, where folks can specify a different runtime per job,  
>> this will be even more critical.
> Sounds like we simply need to create separate jar files for these  
> different components.  This can be done in the current project.
> Wouldn't the amount of effort to make this split and get it right be  
> better spent on getting all components of Hadoop to 1.0 (API  
> stability)?  The proposal feels like a distraction to me at this  
> point in the project.
> Nige

I'd like to retract the -1 vote that I gave this proposal earlier.   
One compelling reason (for me) to split HDFS and Map/Reduce into  
separate sub-projects is that (hopefully) the *configs* for each layer  
will be clearer and simpler.

So I'm now +1 on this proposal.


View raw message