harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: Re: Re: [classlib][TestNG] groups of Harmony test
Date Wed, 13 Sep 2006 12:20:39 GMT
Stepan, All,

> This is not a good argument for using "os.any". Even more it may sound like
> we are going to choose "wrong tool" - why we have to add "os.any" to 99% of
> classlib tests? just because the harness can not do without it?

Well, we have to add the new stuff to these 99% anyway, I mean the
annotation itself. And the copy-pasting of longer block doesn't seem
to be harder than copy-pasting of short block to me.

> As I understand "arguments in the old thread" - TestNG makes us to use "
> os.any" annotation otherwise we have to do a lot of tricks to run tests,
> right?

This was the one of the arguments.

> So a test's annotation should point out that the test is a special one.

I was thinking about a little bit different approach. I will try to
explain. The idea was to use annotations as a *single* place for the
complete information about test: its type, target OS(es) and etc.
Something like inlined test descriptor. Not a red flag. In case
someone is looking at the test at the first time he will get a
complete information just by reading the annotation located at the
same place with the java code. If the annotation itself is not obvious
then the developer is obliged to go to some page with annotation
descriptions, read it carefully and etc. And it seems we get two
places with information about single test: the test's source file and
the page with decoding rules for annotations. This IMHO reduces the
value of the main TestNG benefit - to have all information about the
test in single place. But if we are ok to have several places - we may
use Junit TestSuites instead of TestNG. No refactoring is needed at
all.

To put this another way, these efforts doesn't seem to be so
"unnecessary" to me. Efforts are required to make the life better
sometimes IMHO. Does this make sense to anyone?


Thanks for the reading,

2006/9/12, Stepan Mishura <stepan.mishura@gmail.com>:
> On 9/11/06, Alexei Zakharov  wrote:
> >
> > Hi all,
> >
> > > One more note (seems it already was said sorry if I repeat): the test
> > > without any marks should be run in all configurations (i.e. we have
> > > 'default' group but declaration of this group may be missed).
> >
> > I'd like to point your attention on the previous discussion about
> > "default groups" :
> >
> > http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44C7457B.7080107@googlemail.com%3e
> >
> > I am still for using "os.any" since it is more self-descriptive and
> > the build script will be simpler with "os.any".
>
>
>
> This is not a good argument for using "os.any". Even more it may sound like
> we are going to choose "wrong tool" - why we have to add "os.any" to 99% of
> classlib tests? just because the harness can not do without it?
>
>
> It will be nice to
> > hear more arguments for using defaults because it seems the arguments
> > that were gathered in that old thread hasn't been taken into account
> > by participants of this thread.
>
>
>
> As I understand "arguments in the old thread" - TestNG makes us to use "
> os.any" annotation otherwise we have to do a lot of tricks to run tests,
> right?
>
> IMO a test annotation should be used as "red flag" for developer, for
> example
> * "state.broken" - hey, i'm broken fix me please
> * "os.win" - i'm valid only for Windows, don't try to run me on Linux
>
> So a test's annotation should point out that the test is a special one. But
> if most of tests will contain a similar block of annotations then nobody
> will looked at them. Does this make sense?
>
> Thanks,
> Stepan.
>
> Thanks,
> >
> > 2006/9/5, Vladimir Ivanov <ivavladimir@gmail.com>:
> > > One more note (seems it already was said sorry if I repeat): the test
> > > without any marks should be run in all configurations (i.e. we have
> > > 'default' group but declaration of this group may be missed).
> > >
> > >  thanks, Vladimir
> > >
> > >
> > > On 9/5/06, Vladimir Ivanov <ivavladimir@gmail.com> wrote:
> > > >
> > > >  OK, let's return back to the usage model.
> > > > If I understood it correctly, before the commit of any changes each
> > > > developer run *all* tests (at least all which we have now) on all
> > available
> > > > to him platforms. In this context seems we don't need in any 'level'
> > group
> > > > (while 'stress' tests require reasonable time to pass).
> > > > Seems, that "platform" group also can be deleted (at present time we
> > have
> > > > <10 platform-dependent tests and this amount should not increase
> > > > dramatically so the platform-detection can be included to the each
> > such
> > > > test).
> > > > Also "cpu" groups can be deleted (while we have not cpu-dependent
> > test).
> > > > At the end we need only "state" groups to support test exclusion on
> > the
> > > > 'one-element' level (while we have unresolved entries in the current
> > exclude
> > > > list).
> > > >
> > > > So, after small update of unit (aka integration, aka regression etc)
> > tests
> > > > and resolution of all entries in the exclude list we don't need any
> > groups
> > > > and pure JUnit covers all our needs :)
> > > >
> > > > On the other side, if we define some groups it will nice to define
> > *all*
> > > > reasonable groups at the begin of the process.
> > > >
> > > >  thanks, Vladimir
> > > >
> > > > By the way, our regression tests are 'classic' regression tests that
> > > > demonstrate some issues which were not resolved by initial code. But
> > it
> > > > provides less coverage than 'regression tests' + unit tests, of cause.
> > > >
> > > >  On 9/5/06, Richard Liang <richard.liangyx@gmail.com > wrote:
> > > > >
> > > > > On 9/5/06, Alex Blewitt < alex.blewitt@gmail.com> wrote:
> > > > > > On 04/09/06, Richard Liang <richard.liangyx@gmail.com>
wrote:
> > > > > > > On 9/4/06, Alex Blewitt <alex.blewitt@gmail.com >
wrote:
> > > > > > > >
> > > > > > > > If you've got fast and slow tests, then have a group
for fast
> > and
> > > > > slow
> > > > > > > > tests. Then you can choose to just run the fast tests,
and any
> > > > > > > > automated build system can handle running the slow
tests.
> > > > > > >
> > > > > > > IMHO, "fast or slow" may not be the key point. The question
is
> > > > > whether we
> > > > > > > have any requirements to run only the regression tests.
> > > > > >
> > > > > > No, probably not the key point, but (a) groups don't have to
be
> > > > > > mutually exclusive (so you can decorate it with whatever groups
> > you
> > > > > > want)
> > > > >
> > > > > I agree. For example, os.win and os.linux are not mutually
> > exclusive.
> > > > >
> > > > > Thanks a lot.
> > > > >
> > > > > and (b) it might be useful for an automated build system to run
> > > > > > fast tests first, followed by slow (or non-fast) tests.
> > > > >
> > > > > That makes sense through we have not clear requirement currently.
> > > > >
> > > > > > Mind you, I don't know what's going to happen with an automated
> > > > > test'n'build
> > > > > > system; so it might not make sense to do it at this point.
> > > > >
> > > > > Really? ;-) We could also discuss whether it's feasible to move to
> > > > > TestNG. As you may know, there are already several threads about
> > > > > TestNG & JUnit. Here I just review the open questions one by
one so
> > > > > that we have sufficient preparation.
> > > > >
> > > > >
> > > > > [1]http://mail-
> > archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e
> > > > >
> > > > > [2]http://mail-
> > archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c7273946b0607240654i7e951260x1e803ce476821982@mail.gmail.com%3e
> > > > >
> > > > > [3]http://mail-
> > archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c2c9597b90607280631p2b4f6fefldaf4ff1c5cd00406@mail.gmail.com%3e
> > > > >
> > > > >
> > > > > Best regards,
> > > > > Richard
> > > > >
> > > > > >
> > > > > > Alex.
> > > > > >
> >
> >
> >
> >
>
>
> --
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division


-- 
Alexei Zakharov,
Intel Middleware Product Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message