hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharad Agarwal <sharad.apa...@gmail.com>
Subject Re: MR1 next steps
Date Wed, 27 Jul 2011 05:38:05 GMT
On Tue, Jul 26, 2011 at 4:52 PM, Steve Loughran <stevel@apache.org> wrote:
>
>
> Why does the MiniMR cluster go away in MR2?
>

As Arun pointed out, MiniMR has lot of JT and TT bits ingrained in it which
makes it almost unusable in MR2. I wish tests would have used MiniMR via a
interface then the implementation could be changed to reflect MR2 runtime.

>
>
> These are minor issues. The key thing is that MiniDFS cluster and MiniMR
> cluster let you bring up a short lived MR cluster without any installation
> overhead and run tests against it. Even if the hadoop-mapreduce code can
> move away from it, it's still invaluable downstream
>
>
Absolutely, infact MR2 has got similar things to write effective tests
easily :
- MiniYarnCluster -  includes RM and NM
- MiniMRYarnCluster (extends MiniYarnCluster) - includes AM and MR history
server

These follow Service lifecycles, so much easier to extend and work with.

Thanks
Sharad

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message