harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: Re: Re: [classlib][TestNG] groups of Harmony test
Date Tue, 05 Sep 2006 05:20:31 GMT
On 9/5/06, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
>
> On 9/5/06, Vladimir Ivanov <ivavladimir@gmail.com> wrote:
> >
> > OK, let's return back to the usage model.
> > If I understood it correctly, before the commit of any changes each
> > developer run *all* tests (at least all which we have now) on all
> > available
> > to him platforms.
>
>
> Yes. But as you mentioned, what's "all available"?
>
> If a test passes on windows while fails on linux, is it available to
> windows?
>
> If it is, how will we control it? TestNG groups.


If developer has only windows box than he should run tests on windows only.
If test fails on linux than (for example):
 - the test/ implementation should be fixed or
 - the test should define platform and report 'passed' if it does not
support current platform

In this context seems we don't need in any 'level' group
> > (while 'stress' tests require reasonable time to pass).
> > Seems, that "platform" group also can be deleted (at present time we
> have
> > <10 platform-dependent tests and this amount should not increase
> > dramatically so the platform-detection can be included to the each such
> > test).
> > Also "cpu" groups can be deleted (while we have not cpu-dependent test).
> > At the end we need only "state" groups to support test exclusion on the
> > 'one-element' level (while we have unresolved entries in the current
> > exclude
> > list).
> >
> > So, after small update of unit (aka integration, aka regression etc)
> tests
> > and resolution of all entries in the exclude list we don't need any
> groups
> > and pure JUnit covers all our needs :)
> >
> > On the other side, if we define some groups it will nice to define *all*
> > reasonable groups at the begin of the process.
>
>
> Yes. We should figure out all possible groups. And it can be consolidated
> during applying TestNG.


Agree.
 thanks, Vladimir

thanks, Vladimir
> >
> > By the way, our regression tests are 'classic' regression tests that
> > demonstrate some issues which were not resolved by initial code. But it
> > provides less coverage than 'regression tests' + unit tests, of cause.
> >
> > On 9/5/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> > >
> > > On 9/5/06, Alex Blewitt <alex.blewitt@gmail.com> wrote:
> > > > On 04/09/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> > > > > On 9/4/06, Alex Blewitt <alex.blewitt@gmail.com > wrote:
> > > > > >
> > > > > > If you've got fast and slow tests, then have a group for fast
> and
> > > slow
> > > > > > tests. Then you can choose to just run the fast tests, and any
> > > > > > automated build system can handle running the slow tests.
> > > > >
> > > > > IMHO, "fast or slow" may not be the key point. The question is
> > whether
> > > we
> > > > > have any requirements to run only the regression tests.
> > > >
> > > > No, probably not the key point, but (a) groups don't have to be
> > > > mutually exclusive (so you can decorate it with whatever groups you
> > > > want)
> > >
> > > I agree. For example, os.win and os.linux are not mutually exclusive.
> > >
> > > Thanks a lot.
> > >
> > > and (b) it might be useful for an automated build system to run
> > > > fast tests first, followed by slow (or non-fast) tests.
> > >
> > > That makes sense through we have not clear requirement currently.
> > >
> > > > Mind you, I don't know what's going to happen with an automated
> > > test'n'build
> > > > system; so it might not make sense to do it at this point.
> > >
> > > Really? ;-) We could also discuss whether it's feasible to move to
> > > TestNG. As you may know, there are already several threads about
> > > TestNG & JUnit. Here I just review the open questions one by one so
> > > that we have sufficient preparation.
> > >
> > >
> > > [1]http://mail-
> >
> archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e
> > >
> > > [2]http://mail-
> >
> archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c7273946b0607240654i7e951260x1e803ce476821982@mail.gmail.com%3e
> > >
> > > [3]http://mail-
> >
> archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c2c9597b90607280631p2b4f6fefldaf4ff1c5cd00406@mail.gmail.com%3e
> > >
> > >
> > > Best regards,
> > > Richard
> > >
> > > >
> > > > Alex.
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > >
> > > --
> > > Richard Liang
> > > China Software Development Lab, IBM
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> >
>
>
> --
> Andrew Zhang
> China Software Development Lab, IBM
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message