geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasad Kashyap <goyathlay.geron...@gmail.com>
Subject Re: Test strategy
Date Thu, 09 Feb 2006 21:16:09 GMT
Oops.. sorry. This should have helped.

http://www.mail-archive.com/dev@geronimo.apache.org/msg16726.html

Cheers
Prasad

On 2/9/06, David Blevins <david.blevins@visi.com> wrote:
> On Feb 9, 2006, at 9:10 AM, Prasad Kashyap wrote:
>
> > I have completed items 1 and 3 of the test strategy outlined above.
> > With that we now have the geronimo-deployment-plugin in maven 2.
>
> What are items 1 and 3?  Don't see an outline.
>
> -David
>
>
> > I have now begun working on item 2 which is to create a separate
> > itests sub-project that will act as a catchment of the itests from the
> > various modules. Now this is what we should look into - have the
> > itests subproject as a dependency of the Geronimo project, and include
> > the tests during the integration-phase of the lifecycle in G's build.
> > Yet keep it separate, and thus be able to run the itests against any
> > G's distribution binary separately. This would be the best of both
> > worlds. I'll investigate the possibility of doing so.
> >
> > Jason, it is at this point that I have begun looking at TestNG. I have
> > some questions for you.
> > - Have you used it yourself ?
> > - How do you think it will fit into the picture I painted above where
> > the testsuites will be contributed by folks from the the various
> > modules ?
> > - Do they all have to use TestNG then ?
> >
> > We need a framework that can perform system level tests 'coz that is
> > what itests are at this point.
> >
> > Cheers
> > Prasad
> >
> > On 1/31/06, Jason Dillon <jason.dillon@planet57.com> wrote:
> >> Anyone thought about using TestNG?
> >>
> >> Its xml suite def and grouping support would be nice to define
> >> these itest suites.
> >>
> >> --jason
> >>
> >>
> >> -----Original Message-----
> >> From: David Blevins <david.blevins@visi.com>
> >> Date: Mon, 30 Jan 2006 20:41:39
> >> To:dev@geronimo.apache.org
> >> Subject: Re: Test strategy
> >>
> >>
> >> On Jan 30, 2006, at 7:43 PM, Prasad Kashyap wrote:
> >>> I would like to solicit the views of others too like you and get
> >>> their
> >>> views and opinions.
> >>
> >> Great.  I poked some other people in irc to get involved as well.
> >> Here is hoping.... :)  More input is going to make for a better
> >> community supported solution.
> >>
> >>>>> - The tests can all be run or only a subset of suites can be run
> >>>>> quite easily.
> >>>
> >>>> When you say "easily" do you mean by virtue of them being
> >>>> separate or
> >>>> some other idea?
> >>>
> >>> When I said we could run a subset of tests "easily" when the itests
> >>> are run "on" assemblies, I meant that it would be relatively
> >>> easier to
> >>> selectively include/exclude tests (by categories) than when compared
> >>> to doing the same if the itests are embedded with their modules.
> >>
> >> I think I follow what you mean.  I guess the haze is in what you mean
> >> by "embedded."
> >>
> >> As I'm not sure, I will just state in more specific terms that I have
> >> never argued for putting the actual testing source code inside any
> >> particular assembly module -- this code is big, bulky, has a lot of
> >> deps itself and some pretty specific packaging needs.  I have argued
> >> for simply running the    big test suite during the lifecycle of an
> >> assembly.
> >>
> >> Not sure if that is coming across well, or if it's being assumed that
> >> since I've recommended running the integration tests on an assembly,
> >> during the integration test phase of it's lifecycle, that the test
> >> source of course must be in the assembly as well.  I believe we agree
> >> on keeping the test source in modules separate and organized.
> >>
> >>>>> - The tests will be grouped logically into suites or categories.
> >>>>> Each
> >>>>> individual test will fall in one of these suites.
> >>>
> >>>> Not sure of your usage of the word "test" in this sentence.  In
> >>>> junit
> >>>> terms is that TestSuite, TestCase, or individual test method.  Can
> >>>> you clarify?
> >>>
> >>> I meant TestCase. For. eg, we could have a TestSuite called,
> >>> "System"
> >>> which will have test cases from many different modules. We could say
> >>> that these form the core set of tests. If the System suite
> >>> passed, we
> >>> have somewhat of a stable binary with the caveat that all functions
> >>> may not work.
> >>
> >> That is much clearer to me, thanks.  I do like the sort of parent
> >> module name you came up with of "System" too -- it's a keeper.
> >>
> >> At this point, I will note that it is possible put a single
> >> SuperServerSystemTest.java TestSuite in the SuperServer assemblies
> >> module (for example) that adds only the tests from the many different
> >> test modules that apply to that assembly.
> >>
> >>> In summary, the cons of running the itests "on" assemblies is
> >>> that it
> >>> goes against the m2 lifecycle. And even though integration-test
> >>> is an
> >>> m2 lifecycle phase, we are not exploiting it's usage but calling it
> >>> explicity again. The same has to be done for the deploy phase too.
> >>
> >> You captured that quite well.
> >>
> >> It occurred to me while writing a sentence above that there is no
> >> clear distinction between "on" and "during".  In all cases you are
> >> quite literally running the tests on the assembly.  The better
> >> distinction is whether or not you wish to run them "during" it's
> >> lifecycle or "after" in the lifecycle of a separate module.
> >>
> >> -David
> >>
> >
>
>

Mime
View raw message