harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Zhang" <zhanghuang...@gmail.com>
Subject Re: Re: Re: [classlib][TestNG] groups of Harmony test
Date Tue, 05 Sep 2006 05:11:37 GMT
On 9/5/06, Vladimir Ivanov <ivavladimir@gmail.com> wrote:
>
> OK, let's return back to the usage model.
> If I understood it correctly, before the commit of any changes each
> developer run *all* tests (at least all which we have now) on all
> available
> to him platforms.


Yes. But as you mentioned, what's "all available"?

If a test passes on windows while fails on linux, is it available to
windows?

If it is, how will we control it? TestNG groups.

 In this context seems we don't need in any 'level' group
> (while 'stress' tests require reasonable time to pass).
> Seems, that "platform" group also can be deleted (at present time we have
> <10 platform-dependent tests and this amount should not increase
> dramatically so the platform-detection can be included to the each such
> test).
> Also "cpu" groups can be deleted (while we have not cpu-dependent test).
> At the end we need only "state" groups to support test exclusion on the
> 'one-element' level (while we have unresolved entries in the current
> exclude
> list).
>
> So, after small update of unit (aka integration, aka regression etc) tests
> and resolution of all entries in the exclude list we don't need any groups
> and pure JUnit covers all our needs :)
>
> On the other side, if we define some groups it will nice to define *all*
> reasonable groups at the begin of the process.


Yes. We should figure out all possible groups. And it can be consolidated
during applying TestNG.

thanks, Vladimir
>
> By the way, our regression tests are 'classic' regression tests that
> demonstrate some issues which were not resolved by initial code. But it
> provides less coverage than 'regression tests' + unit tests, of cause.
>
> On 9/5/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> >
> > On 9/5/06, Alex Blewitt <alex.blewitt@gmail.com> wrote:
> > > On 04/09/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> > > > On 9/4/06, Alex Blewitt <alex.blewitt@gmail.com > wrote:
> > > > >
> > > > > If you've got fast and slow tests, then have a group for fast and
> > slow
> > > > > tests. Then you can choose to just run the fast tests, and any
> > > > > automated build system can handle running the slow tests.
> > > >
> > > > IMHO, "fast or slow" may not be the key point. The question is
> whether
> > we
> > > > have any requirements to run only the regression tests.
> > >
> > > No, probably not the key point, but (a) groups don't have to be
> > > mutually exclusive (so you can decorate it with whatever groups you
> > > want)
> >
> > I agree. For example, os.win and os.linux are not mutually exclusive.
> >
> > Thanks a lot.
> >
> > and (b) it might be useful for an automated build system to run
> > > fast tests first, followed by slow (or non-fast) tests.
> >
> > That makes sense through we have not clear requirement currently.
> >
> > > Mind you, I don't know what's going to happen with an automated
> > test'n'build
> > > system; so it might not make sense to do it at this point.
> >
> > Really? ;-) We could also discuss whether it's feasible to move to
> > TestNG. As you may know, there are already several threads about
> > TestNG & JUnit. Here I just review the open questions one by one so
> > that we have sufficient preparation.
> >
> >
> > [1]http://mail-
> archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e
> >
> > [2]http://mail-
> archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c7273946b0607240654i7e951260x1e803ce476821982@mail.gmail.com%3e
> >
> > [3]http://mail-
> archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c2c9597b90607280631p2b4f6fefldaf4ff1c5cd00406@mail.gmail.com%3e
> >
> >
> > Best regards,
> > Richard
> >
> > >
> > > Alex.
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
> > --
> > Richard Liang
> > China Software Development Lab, IBM
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>


-- 
Andrew Zhang
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message