On 2/5/07, Alex Karasulu <akarasulu@apache.org > wrote:

---------- Forwarded message ----------
Emmanuel Lecharny wrote:
> Yeah, running integration tests is time consumming. For 1.0.1, it's
> acceptable, as it takes only 3min (depending on your computer), but in
> 1.5, it's around 5 times slower.

We have many more tests about 40% more.

Yeah. In 1.0 we have 218 tests in core-unit, and 365 in 1.5

> We may consider finding out why it takes so long in 1.5 (server
> initialization takes around 10 to 15 seconds, which is not really normal)

Please Emmanuel it does not take this long for startup. You're going to
freak people out by saying stuff like this. Let's watch these kinds of
exaggerations.

Sorry, I was totally wrong about those numbers. I guess I pulled them out of my head just when I woke up this morning, after a very bad sleep.

Let me correct them :
- each unit test is around 1 sec, startup and shutdown included in 1.5.0, not 10 to 15 seconds.
- in 1.0.1, each test is around 0,2 seconds, startup and shutdown included.

Be aware that those number has been taken on *my* computer, and by no means they are the official numbers of Apache Directory Server.

It does take a little longer in 1.5 but really we have about 40% more
tests and initialization time is slower but no where near 10-15 seconds.

A single integration test case (with one test method) takes about 3.5
seconds to start, stop and cleanup on my old hardware (AMD 2800+).

It's faster on my computer. 3.5 seconds is really above what I really get. I got around 1.8 to 2.5 seconds for single tests, but you have to ake into account the classloading of the unit test and stuff like that, which is not free...

Here's a run of just one empty test case. Committed here:

mvn -Dtest=OneEmptyITest -Dintegration test
Running org.apache.directory.server.core.schema.OneEmptyITest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.538 sec

I get better results :)

Running org.apache.directory.server.core.schema.OneEmptyITest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.929 sec

Man, my imac is twice faster than your old AMD ;)

And as you can see 3.538 seconds is not anywhere near 10-15 seconds
exaggeration.

Very true. My bad ... Never ever drink on sunday evening ;(

Now you said that this is 5x slower in 1.5 branch than it is in the 1.0
branch.
So we're only 2x slower in the 1.5 branch than in the 1.0 branch. Again
that's way less than the 5 times figure you just threw out there.

You are doing micro benchmark here. What I was talking about was the whole Integration test. It takes 2 mins and a half on 1.0.1, and 12 mins 06 seconds on 1.5.0
If you eliminate all the artefacts, it's around 4 times slower, considering we have 40% more tests...