Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:from:to:in-reply-to:content-type:
	content-transfer-encoding:mime-version:subject:date:references:x-mailer;
	b=AbpEuZyXo18R0NoXhrN5QDlw6tk/3Gy0aNleBXNXYzW1+t0QfWfPHLTalHQcVKaA
Message-Id: <175CD9D8-34A3-4A66-87B0-359EA5E22169@yahoo-inc.com>
From: Nigel Daley <ndaley@yahoo-inc.com>
To: general@hadoop.apache.org
In-Reply-To: <489CA74F.8000901@apache.org>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v928.1)
Subject: Re: [VOTE] Should we create sub-projects for HDFS and Map/Reduce?
Date: Fri, 15 Aug 2008 22:03:10 -0700
References: <688CEB2A-13EF-4C38-855A-3D6D4C3485B3@yahoo-inc.com>
 <2DA5A7B1-0CC0-4C4F-BE80-AE1E3C7B47A3@yahoo-inc.com>
 <489CA74F.8000901@apache.org>

> Another benefit is that it would increase the separation of these  
> technologies, so that, e.g., folks could more easily run different  
> versions of mapreduce on top of different versions of HDFS.   
> Currently we make no such guarantees.  Folks would be able to  
> upgrade to, e.g., the next release of mapreduce on a subset of their  
> cluster without upgrading their HDFS.  That's not currently  
> supported.  As we move towards splitting mapreduce into a scheduler  
> and runtime, where folks can specify a different runtime per job,  
> this will be even more critical.

Sounds like we simply need to create separate jar files for these  
different components.  This can be done in the current project.

Wouldn't the amount of effort to make this split and get it right be  
better spent on getting all components of Hadoop to 1.0 (API  
stability)?  The proposal feels like a distraction to me at this point  
in the project.

Nige