harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Liang" <richard.lian...@gmail.com>
Subject Re: Re: Re: [classlib][TestNG] groups of Harmony test
Date Wed, 13 Sep 2006 01:45:57 GMT
On 9/12/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> On 9/11/06, Alexei Zakharov  wrote:
> >
> > Hi all,
> >
> > > One more note (seems it already was said sorry if I repeat): the test
> > > without any marks should be run in all configurations (i.e. we have
> > > 'default' group but declaration of this group may be missed).
> >
> > I'd like to point your attention on the previous discussion about
> > "default groups" :
> >
> > http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44C7457B.7080107@googlemail.com%3e
> >
> > I am still for using "os.any" since it is more self-descriptive and
> > the build script will be simpler with "os.any".
>
>
>
> This is not a good argument for using "os.any". Even more it may sound like
> we are going to choose "wrong tool" - why we have to add "os.any" to 99% of
> classlib tests? just because the harness can not do without it?
>

Yes. And we have found a way to avoid "os.any" ;-)

>
> It will be nice to
> > hear more arguments for using defaults because it seems the arguments
> > that were gathered in that old thread hasn't been taken into account
> > by participants of this thread.
>
>
>
> As I understand "arguments in the old thread" - TestNG makes us to use "
> os.any" annotation otherwise we have to do a lot of tricks to run tests,
> right?
>
> IMO a test annotation should be used as "red flag" for developer, for
> example
> * "state.broken" - hey, i'm broken fix me please
> * "os.win" - i'm valid only for Windows, don't try to run me on Linux
>
> So a test's annotation should point out that the test is a special one. But
> if most of tests will contain a similar block of annotations then nobody
> will looked at them. Does this make sense?

Yes. It really makes sense. We do not want to involve developers in
unnecessary effort.

>
> Thanks,
> Stepan.
>
> Thanks,
> >
> > 2006/9/5, Vladimir Ivanov <ivavladimir@gmail.com>:
> > > One more note (seems it already was said sorry if I repeat): the test
> > > without any marks should be run in all configurations (i.e. we have
> > > 'default' group but declaration of this group may be missed).
> > >
> > >  thanks, Vladimir
> > >
> > >
> > > On 9/5/06, Vladimir Ivanov <ivavladimir@gmail.com> wrote:
> > > >
> > > >  OK, let's return back to the usage model.
> > > > If I understood it correctly, before the commit of any changes each
> > > > developer run *all* tests (at least all which we have now) on all
> > available
> > > > to him platforms. In this context seems we don't need in any 'level'
> > group
> > > > (while 'stress' tests require reasonable time to pass).
> > > > Seems, that "platform" group also can be deleted (at present time we
> > have
> > > > <10 platform-dependent tests and this amount should not increase
> > > > dramatically so the platform-detection can be included to the each
> > such
> > > > test).
> > > > Also "cpu" groups can be deleted (while we have not cpu-dependent
> > test).
> > > > At the end we need only "state" groups to support test exclusion on
> > the
> > > > 'one-element' level (while we have unresolved entries in the current
> > exclude
> > > > list).
> > > >
> > > > So, after small update of unit (aka integration, aka regression etc)
> > tests
> > > > and resolution of all entries in the exclude list we don't need any
> > groups
> > > > and pure JUnit covers all our needs :)
> > > >
> > > > On the other side, if we define some groups it will nice to define
> > *all*
> > > > reasonable groups at the begin of the process.
> > > >
> > > >  thanks, Vladimir
> > > >
> > > > By the way, our regression tests are 'classic' regression tests that
> > > > demonstrate some issues which were not resolved by initial code. But
> > it
> > > > provides less coverage than 'regression tests' + unit tests, of cause.
> > > >
> > > >  On 9/5/06, Richard Liang <richard.liangyx@gmail.com > wrote:
> > > > >
> > > > > On 9/5/06, Alex Blewitt < alex.blewitt@gmail.com> wrote:
> > > > > > On 04/09/06, Richard Liang <richard.liangyx@gmail.com>
wrote:
> > > > > > > On 9/4/06, Alex Blewitt <alex.blewitt@gmail.com >
wrote:
> > > > > > > >
> > > > > > > > If you've got fast and slow tests, then have a group
for fast
> > and
> > > > > slow
> > > > > > > > tests. Then you can choose to just run the fast tests,
and any
> > > > > > > > automated build system can handle running the slow
tests.
> > > > > > >
> > > > > > > IMHO, "fast or slow" may not be the key point. The question
is
> > > > > whether we
> > > > > > > have any requirements to run only the regression tests.
> > > > > >
> > > > > > No, probably not the key point, but (a) groups don't have to
be
> > > > > > mutually exclusive (so you can decorate it with whatever groups
> > you
> > > > > > want)
> > > > >
> > > > > I agree. For example, os.win and os.linux are not mutually
> > exclusive.
> > > > >
> > > > > Thanks a lot.
> > > > >
> > > > > and (b) it might be useful for an automated build system to run
> > > > > > fast tests first, followed by slow (or non-fast) tests.
> > > > >
> > > > > That makes sense through we have not clear requirement currently.
> > > > >
> > > > > > Mind you, I don't know what's going to happen with an automated
> > > > > test'n'build
> > > > > > system; so it might not make sense to do it at this point.
> > > > >
> > > > > Really? ;-) We could also discuss whether it's feasible to move to
> > > > > TestNG. As you may know, there are already several threads about
> > > > > TestNG & JUnit. Here I just review the open questions one by
one so
> > > > > that we have sufficient preparation.
> > > > >
> > > > >
> > > > > [1]http://mail-
> > archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e
> > > > >
> > > > > [2]http://mail-
> > archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c7273946b0607240654i7e951260x1e803ce476821982@mail.gmail.com%3e
> > > > >
> > > > > [3]http://mail-
> > archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c2c9597b90607280631p2b4f6fefldaf4ff1c5cd00406@mail.gmail.com%3e
> > > > >
> > > > >
> > > > > Best regards,
> > > > > Richard
> > > > >
> > > > > >
> > > > > > Alex.
> > > > > >
> >
> >
> >
> >
>
>
> --
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
> ------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Richard Liang
China Development Lab, IBM

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message