On Mon, Feb 9, 2009 at 11:52 AM, Emmanuel Lecharny <firstname.lastname@example.org>
the cool annotations we have, combined with the rollback system,
allows us to run tests without any problem. Each test can play with
data without interacting with the other tests. However, as the LDIF
injection is done globally for each test, the rollback is also done
globally. ie, if you inject a lot of data, you will have to revert a
lot of data too, for each single test.
We are using it today for one of our client, and as we have around 400
entries to inject for our tests, it takes more or less a couple of
seconds to run each test. At the end of the day, when we run our 150
tests, we have to wait 10 minutes to run all of them.
Use suites and have a full cleanup take place between the suites. You have various cleanup levels (test level, class level, suite level etc) and you can set it up so that a suite class has an annotation which make it cleanup when the suite is started with a PRISTINE server. That's why all this was put in there. We use this today for the Apache DS tests.
No need to tell that it's way faster ! I was able to run the
SearchITest in less than 5 seconds compared to the usual 15 seconds
for the very same tests (and i'm just talking of 4 entries injected
and rollbacked for every tests).
So I think this is the way to go. The only difficult point is that we
have to deal with 5 different modes for tests :
- ROLLBACK (the default)
To me, the RESTART, CUMULATIVE and NOSERVICE mode seems a bit useless.
There are reasons for these - I can explain this tomorrow. Please don't nix this stuff until we discuss it somemore.
The PRISTINE mode is used for one single test in which we declare two
different factories, hence the need to clean up completely the server.
I would suggest we get rid of the RESTART, CUMULATIVE and NOSERVICE
mode in order to ease the implementation of my small improvement
(otehriwse, there are too many cases to deal with, and i don't realy
have time for that).