directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ole Ersoy <ole_er...@yahoo.com>
Subject Re: [Testing] Coping with so many integration tests
Date Mon, 05 Feb 2007 15:29:15 GMT
Emmanuel,

Good idea.

I'll try to get that into the testing document 
under something like "Various testing approaches".

WRT to nightly builds, I probably should have used
different terminology.

How about I call it "The CPU Black Hole Build".

So it would be performed on Continuum or Cruise
Control whenever something is checked into subversion.

Now the Subversion for the "Black Hole Build" could be
different from the Apache Subversion.  It's a
"Continuous Integration" Subversion.  We do the
continuous integration buidls there, and then when
they are green, the code is allowed into the main
subversion trunk.

Cheers,
- Ole


--- Emmanuel Lecharny <elecharny@gmail.com> wrote:

> On 2/4/07, Alex Karasulu <akarasulu@apache.org>
> wrote:
> >
> > Hi,
> >
> > The number of integration tests are growing large
> (we have hundreds) as
> > we get more coverage for the server and the
> functionality increases.
> > These tests start and stop apacheds and clean out
> working directories on
> > each run so they can take a while especially if a
> test case has lots of
> > tests in it.
> >
> > I have two ideas on how to ameliorate these
> problems.  One is short term
> > and the other long term.
> >
> > First the short term solution.  I found that I can
> create large 512K ram
> > drives (do not try more than 512K it does not
> work) then combine them
> > together using RAID to make a large disk.  After
> having done this test
> > case classes that took 20 seconds started to take
> 4 seconds instead.
> > Not bad but still not ideal since we have so many
> tests.  This however
> > does not have that many drawbacks besides still
> taking 9 minutes and 49
> > seconds to run all the integration tests.
> >
> > Long term solution:  See if we can modify the
> surefire plugin for
> > distributed operation.  It would be easy to have a
> java daemon that can
> > run maven on various projects.  Commands can be
> sent to this to setup
> > and run individual test case classes.  The
> distributed surefire plugin
> > would simply queue operations and collect the
> results from several
> > machines.  This way you get serious parallelism
> especially if you have
> > many machines.  With four machines the time could
> be cut down to 2 and a
> > half minutes.
> >
> > Thoughts?
> 
> 
> Yeah, running integration tests is time consumming.
> For 1.0.1, it's
> acceptable, as it takes only 3min (depending on your
> computer), but in 1.5,
> it's around 5 times slower.
> 
> We may consider finding out why it takes so long in
> 1.5 (server
> initialization takes around 10 to 15 seconds, which
> is not really normal)
> 
> However, even with shortest tests, it's still quite
> long, and it won't be
> better in the next few months (i'm not sure that the
> increase rate of tests
> is lower that Intel processor performance increase
> rate ... :)
> 
> May I suggest a third solution? We might use tools
> like jmeter which are
> able to launch tests against a ldap server, playing
> full scenarios. The good
> part of it is that you can prepare the scenarios
> with a GUI, and save them
> as XML files to be replayed within maven (there is a
> maven-jmeter-plugin).
> The good points about this scenario that you only
> launch the server once,
> tests are easy to write and to maintain, and there
> is no more relation with
> the actual code base, so no huge refactoring to do
> in tests when modifying
> base classes like AttributeImpl.
> 
> 2cts...
> 
> PS : Ole, depending on nighty builds is not an
> option. When you are working
> on trunks, you should *not* commit something that
> break the server. Nighty
> Builds are just a safety belt, because sometime you
> just break this rule,
> without knowing it (damn local repo ...)
> 
> Alex
> >
> >
> 
> 
> -- 
> Cordialement,
> Emmanuel L├ęcharny
> www.iktek.com
> 



 
____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

Mime
View raw message