Emmanuel Lecharny wrote:
> Yeah, running integration tests is time consumming. For 1.0.1, it's
> acceptable, as it takes only 3min (depending on your computer), but in
> 1.5, it's around 5 times slower.
We have many more tests about 40% more.
> We may consider finding out why it takes so long in 1.5 (server
> initialization takes around 10 to 15 seconds, which is not really normal)
Please Emmanuel it does not take this long for startup. You're going to
freak people out by saying stuff like this. Let's watch these kinds of
exaggerations.
It does take a little longer in 1.5 but really we have about 40% more
tests and initialization time is slower but no where near 10-15 seconds.
A single integration test case (with one test method) takes about 3.5
seconds to start, stop and cleanup on my old hardware (AMD 2800+).
Here's a run of just one empty test case. Committed here:
http://svn.apache.org/viewvc?view=rev&revision=503758
Here's how to run it in apacheds/core-unit
mvn -Dtest=OneEmptyITest -Dintegration test
Here's how much time it took for this ITest to run:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.directory.server.core.schema.OneEmptyITest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.538 sec
And as you can see 3.538 seconds is not anywhere near 10-15 seconds
exaggeration. Now if you're running this on a really slow machine (400
Mhz) with bad drives then I can see this being the case.
Now you said that this is 5x slower in 1.5 branch than it is in the 1.0
branch. Let's take a look at the same test case in the 1.0 branch.
Here's there result there:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.directory.server.core.schema.OneEmptyITest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.752 sec
So we're only 2x slower in the 1.5 branch than in the 1.0 branch. Again
that's way less than the 5 times figure you just threw out there.
Please let's be careful with what we toss out there next time. People
are listening and will run with negative news especially when it's not
true and is coming from the mouths of our own people.
Brotha man, I hope you understand my reasons for asking this and don't
think its me trying to bust your keyboard :).
...
> May I suggest a third solution? We might use tools like jmeter which are
> able to launch tests against a ldap server, playing full scenarios. The
> good part of it is that you can prepare the scenarios with a GUI, and
> save them as XML files to be replayed within maven (there is a
> maven-jmeter-plugin). The good points about this scenario that you only
> launch the server once, tests are easy to write and to maintain, and
> there is no more relation with the actual code base, so no huge
> refactoring to do in tests when modifying base classes like AttributeImpl.
This is a great idea but how do we get this to be part of the build
process? Basically how do we get maven to launch a test server on
startup that will be available for the entire run while tests operate on it?
This is a great idea if we can make maven do it.
Alex
|