hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhruba Borthakur" <dhr...@gmail.com>
Subject Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?
Date Mon, 15 Sep 2008 22:03:06 GMT

I would prefer to keep hdfs and mapreduce together because I believe
that this arrangement catches incompatability sooner that later. But
with the coming of age of both these modules separately, I guess it is
time to grow them on their own individual turf!


On Fri, Sep 5, 2008 at 5:21 PM, Nigel Daley <ndaley@yahoo-inc.com> wrote:
> On Aug 15, 2008, at 10:03 PM, Nigel Daley wrote:
>>> Another benefit is that it would increase the separation of these
>>> technologies, so that, e.g., folks could more easily run different versions
>>> of mapreduce on top of different versions of HDFS.  Currently we make no
>>> such guarantees.  Folks would be able to upgrade to, e.g., the next release
>>> of mapreduce on a subset of their cluster without upgrading their HDFS.
>>>  That's not currently supported.  As we move towards splitting mapreduce
>>> into a scheduler and runtime, where folks can specify a different runtime
>>> per job, this will be even more critical.
>> Sounds like we simply need to create separate jar files for these
>> different components.  This can be done in the current project.
>> Wouldn't the amount of effort to make this split and get it right be
>> better spent on getting all components of Hadoop to 1.0 (API stability)?
>>  The proposal feels like a distraction to me at this point in the project.
>> Nige
> I'd like to retract the -1 vote that I gave this proposal earlier.  One
> compelling reason (for me) to split HDFS and Map/Reduce into separate
> sub-projects is that (hopefully) the *configs* for each layer will be
> clearer and simpler.
> So I'm now +1 on this proposal.
> Nige

View raw message