hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: MR1 next steps
Date Wed, 27 Jul 2011 09:39:13 GMT
On 27/07/11 06:38, Sharad Agarwal wrote:
> On Tue, Jul 26, 2011 at 4:52 PM, Steve Loughran<stevel@apache.org>  wrote:
>> Why does the MiniMR cluster go away in MR2?
> As Arun pointed out, MiniMR has lot of JT and TT bits ingrained in it which
> makes it almost unusable in MR2. I wish tests would have used MiniMR via a
> interface then the implementation could be changed to reflect MR2 runtime.
>> These are minor issues. The key thing is that MiniDFS cluster and MiniMR
>> cluster let you bring up a short lived MR cluster without any installation
>> overhead and run tests against it. Even if the hadoop-mapreduce code can
>> move away from it, it's still invaluable downstream
> Absolutely, infact MR2 has got similar things to write effective tests
> easily :
> - MiniYarnCluster -  includes RM and NM
> - MiniMRYarnCluster (extends MiniYarnCluster) - includes AM and MR history
> server
> These follow Service lifecycles, so much easier to extend and work with.
> Thanks
> Sharad

As long as there is an in-vm mini cluster for testing I'm happy. I am 
building up a list of things to fix in MiniDFS cluster, which is going 
to stay as is, right?

View raw message