Thanks for this Emm. BTW I think CUMULATIVE was put in so if one test set something up the other can use it. But this situation is seldom if at all encountered.
following all the mails about the tests, here is a small analysis I
have doen regarding the Mode used and the consequences on the tests.
I'm considering only three modes here :
- RESTART : the server will be restarted but the base will remain on
the disk (usefull when trying to see if we can read the data after
having stopped the server, or for replication)
- PRISTINE : the server will be stopped, data will be cleaned and the
server will be restarted
- ROLLBACK : the changes will be rollbacked, the server remains started
The other mode (CUMMULATIVE and NOSERVICE) are not included in this
small study as the CUMMULATIVE seems to be useless, as we have no way
to guarantee the test orders and for NOSERVICE, as we don't care about
the directory service being up or down, we won't do anything special.
We have three cleanup levels : SUITE, CLASS and METHOD
We have three Modes : RESTART, PRISTINE and ROLLBACK
let's see what we will have to do for each combinaison of those modes
and cleanup levels, assuming we can also avoid declaring a cleanup
level, hence defining a forth cleanup level, which is the inherited
suite class method action
N/A N/A restart server shutdown and restarted for each test
N/A N/A pristine server shutdown, cleaned and started
for each test
restart retstart rollback server shutdown and restarted for every class
restart retstart N/A server shutdown and restarted for every class
restart pristine rollback server shutdown, cleanup and restart for every class
restart pristine N/A server shutdown, cleanup and restart for
everything else : server started once
As we can see, if the Method has a RESTART or PRISTINE mode, we don't
care about the SUITE and CLASS mode. On the other side, if the Method
does not define a mode or is using ROLLBACK, we will restart the
server accordingly to the Class mode if it's PRISTINE or RESTART, and
last, not least, in any other case, we start the server once, and run
each test with a rollback the test's modifications after each test.
More to come...