hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: bringing the codebases back in line
Date Fri, 22 Oct 2010 16:36:55 GMT
On Fri, Oct 22, 2010 at 12:40PM, Steve Loughran wrote:
> On 22/10/10 01:10, Konstantin Boudnik wrote:
> >
> >The only way, IMO, to have a reasonable testing done on a system as complex as
> >Hadoop is to invest into automatic validation of builds at system level. This
> >requires a few things (resources, if you will):
> >   - extra hardware (the easiest and cheapest problem)
> >   - automatic deployment, testing, and analysis
> >   - system tests development which able to control and observe a cluster
> >     behavior (in other words something more sophisticated than just shell
> >     scripts)
> >
> +1 for testing, I would like to help with this, but my test stuff
> depends on my lifecycle stuff which I need to sit down, sync up with
> trunk and work out how to get in.
> One thing you can do in a virtual world which you can't do in the
> physical space is reconfigure the LAN on the fly, to see what
> happens. For example, I could set up VLANs of two racks and a switch
> between them, then partition the two  and see what happens -while a
> simulated external load (separate issue) hits the NN with the same
> amount of traffic. Fun things.

Awesome idea! I guess it is well aligned with Herriot's abilities to do fault
injections on real (or virual) hardware.

View raw message