geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Blevins <david.blev...@visi.com>
Subject Re: Test strategy
Date Tue, 31 Jan 2006 04:41:39 GMT

On Jan 30, 2006, at 7:43 PM, Prasad Kashyap wrote:
> I would like to solicit the views of others too like you and get their
> views and opinions.

Great.  I poked some other people in irc to get involved as well.   
Here is hoping.... :)  More input is going to make for a better  
community supported solution.

>>> - The tests can all be run or only a subset of suites can be run
>>> quite easily.
>
>> When you say "easily" do you mean by virtue of them being separate or
>> some other idea?
>
> When I said we could run a subset of tests "easily" when the itests
> are run "on" assemblies, I meant that it would be relatively easier to
> selectively include/exclude tests (by categories) than when compared
> to doing the same if the itests are embedded with their modules.

I think I follow what you mean.  I guess the haze is in what you mean  
by "embedded."

As I'm not sure, I will just state in more specific terms that I have  
never argued for putting the actual testing source code inside any  
particular assembly module -- this code is big, bulky, has a lot of  
deps itself and some pretty specific packaging needs.  I have argued  
for simply running the    big test suite during the lifecycle of an  
assembly.

Not sure if that is coming across well, or if it's being assumed that  
since I've recommended running the integration tests on an assembly,  
during the integration test phase of it's lifecycle, that the test  
source of course must be in the assembly as well.  I believe we agree  
on keeping the test source in modules separate and organized.

>>> - The tests will be grouped logically into suites or categories.  
>>> Each
>>> individual test will fall in one of these suites.
>
>> Not sure of your usage of the word "test" in this sentence.  In junit
>> terms is that TestSuite, TestCase, or individual test method.  Can
>> you clarify?
>
> I meant TestCase. For. eg, we could have a TestSuite called, "System"
> which will have test cases from many different modules. We could say
> that these form the core set of tests. If the System suite passed, we
> have somewhat of a stable binary with the caveat that all functions
> may not work.

That is much clearer to me, thanks.  I do like the sort of parent  
module name you came up with of "System" too -- it's a keeper.

At this point, I will note that it is possible put a single  
SuperServerSystemTest.java TestSuite in the SuperServer assemblies  
module (for example) that adds only the tests from the many different  
test modules that apply to that assembly.

> In summary, the cons of running the itests "on" assemblies is that it
> goes against the m2 lifecycle. And even though integration-test is an
> m2 lifecycle phase, we are not exploiting it's usage but calling it
> explicity again. The same has to be done for the deploy phase too.

You captured that quite well.

It occurred to me while writing a sentence above that there is no  
clear distinction between "on" and "during".  In all cases you are  
quite literally running the tests on the assembly.  The better  
distinction is whether or not you wish to run them "during" it's  
lifecycle or "after" in the lifecycle of a separate module.

-David


Mime
View raw message