hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharad Agarwal <sharad.apa...@gmail.com>
Subject Re: MR1 next steps
Date Wed, 27 Jul 2011 05:38:05 GMT
On Tue, Jul 26, 2011 at 4:52 PM, Steve Loughran <stevel@apache.org> wrote:
> Why does the MiniMR cluster go away in MR2?

As Arun pointed out, MiniMR has lot of JT and TT bits ingrained in it which
makes it almost unusable in MR2. I wish tests would have used MiniMR via a
interface then the implementation could be changed to reflect MR2 runtime.

> These are minor issues. The key thing is that MiniDFS cluster and MiniMR
> cluster let you bring up a short lived MR cluster without any installation
> overhead and run tests against it. Even if the hadoop-mapreduce code can
> move away from it, it's still invaluable downstream
Absolutely, infact MR2 has got similar things to write effective tests
easily :
- MiniYarnCluster -  includes RM and NM
- MiniMRYarnCluster (extends MiniYarnCluster) - includes AM and MR history

These follow Service lifecycles, so much easier to extend and work with.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message