harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib][TestNG] groups of Harmony test
Date Mon, 04 Sep 2006 05:10:28 GMT
Hi Vladimir

Could you please decribe for what purpose it will be used?

I mean why one might have to either exclude or run only regression tests?

Thanks,
Mikhail

2006/8/30, Vladimir Ivanov <ivavladimir@gmail.com>:
> On 8/30/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> >
> >
> >
> > Vladimir Ivanov wrote:
> > > Also some tag for regression tests should be added.
> > Yes. Do you think we could annotate regression test as
> > *level.regression*? Thanks a lot.
>
>
> Yes, I do. While tests can have more than one group it will enough.
>  thanks, Vladimir
>
>
> Richard
> > > thanks, Vladimir
> > >
> > >
> > > On 8/28/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> > >>
> > >>
> > >>
> > >> Richard Liang wrote:
> > >> > Hello All,
> > >> >
> > >> > Now let's talk about the TestNG groups. I have read the related
> > >> > threads which posted by George, Vladimir Ivanov and Alexei Zakharov.
> > >> > All of them are good discussion about TestNG groups.
> > >> >
> > >> > IMHO, we may define Harmony test groups according the following 4
> > >> > dimensions:
> > >> >
> > >> > 1) [Platform] os.any, os.<platform id>
> > >> > *os.any* - group of tests which pass on any platform. IMHO, most of
> > >> > our tests should be in this group.
> > >> > *os.<platform id>* - group of tests which are designed for one
> > >> > specific platform. A test may be in more than one of the groups. e.g
> > .,
> > >> > @Test(groups={"os.win.IA32", "os.linux.IA32"})
> > >> >
> > >> >    ** os.any and os.<platform id> are mutually exclusive, that
is,
> > >> > tests in os.any group should not be in os.win.IA32.
> > >> >
> > >> > 2) [Test state] state.broken, state.broken.<platform id>
> > >> > *state.broken* - group of tests which fail on every platform, because
> > >> > of bugs of tests or implementation. We need to fix the bugs of tests
> > >> > or implementation to make them pass.
> > >> > *state.broken.<platform id>* - groups of test which only fail
on one
> > >> > specific platform. A test may be in more than one of the groups. e.g
> > .,
> > >> > @Test(groups={"state.broken.linux.IA32", "os.broken.linux.IA64"})
> > >> >
> > >> >     **state.broken.<platform id> group may be used as a convenient
> > way
> > >> > to indicate that a test is platform-specific. e.g., If we support
10
> > >> > platforms, and one test are designed for 9 platforms except for
> > MacOS,
> > >> > instead of list 9 os.<platform id>, we can just use
> > state.broken.MacOS
> > >> >
> > >> > 3) [Test type] type.api , type.impl
> > >> > *type.api* - group of tests which are tests for APIs in the Java
> > >> > Specification
> > >> > *type.impl* - groups of tests which are tests for Harmony-specific
> > >> > implementation
> > >> >
> > >> >     ** type.api and type.impl are also mutually exclusive.
> > >> >
> > >> > 4) [Test Level] level.unit, level.integration, level.system,
> > >> > level.stress, etc. (Levels of Test refer to the increase in
> > complexity
> > >> > as moving through test cycle. )
> > >> >    ** A test may be in more than one of the groups.
> > >> >    ** In fact, some tests such as System tests are the verification
> > of
> > >> > the entire system.  Maybe we'll put them into a separate project.
> > >> > e.g., harmony/enhanced/SVT (System Verification Test).
> > >> >
> > >> > If we want to run all the unit test for APIs on windows, we may use
> > >> > TestNG groups to select the tests:
> > >> >        <groups>
> > >> >            <run>
> > >> >                <include name="os.any" />
> > >> >                <include name="type.api" />
> > >> >                <include name="os.win.IA32" />
> > >> >                <exclude name=" state.broken" />
> > >> >                <exclude name="state.broken.win.IA32" />
> > >> >            </run>
> > >> >        </groups>
> > >> >
> > >> Hello All,
> > >>
> > >> I'm sorry. It seems that the example does not work. I will try to
> > figure
> > >> another example soon. ;-)
> > >>
> > >> Best regards,
> > >> Richard
> > >> >
> > >> > Well, I think our most of existing tests are in the groups of
> > >> > {"os.any", "type.api", "level.unit"}, and I have asked TestNG to add
> > a
> > >> > new option "-groups" for its JUnitConverter which allow us to specify
> > >> > the test groups when migrate from JUnit test to TestNG test.
> > >> >
> > >> > Thanks for reading so far, and I will highly appreciate your comments
> > >> > or suggestion.  ;-)
> > >> >
> > >>
> > >> --
> > >> Richard Liang
> > >> China Software Development Lab, IBM
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> > >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >>
> > >>
> > >
> >
> > --
> > Richard Liang
> > China Software Development Lab, IBM
> >
> >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message