harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib][TestNG] groups of Harmony test
Date Mon, 04 Sep 2006 10:30:24 GMT
As we agreed earlier [1], for information reasons we include comment
// Regression for HARMONY-xxxx

Thanks,
Mikhail

[1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/%3c6e47b64f0603280346u3dc491edp8f06bacdbd04c626@mail.gmail.com%3e

2006/9/4, Ivanov, Alexey A <alexey.a.ivanov@intel.com>:
>
> >-----Original Message-----
> >From: Richard Liang [mailto:richard.liangyx@gmail.com]
> >Sent: Monday, September 04, 2006 10:56 AM
> >To: harmony-dev@incubator.apache.org
> >Subject: Re: [classlib][TestNG] groups of Harmony test
> >
> >
> >
> >Mikhail Loenko wrote:
> >> Hi Vladimir
> >>
> >> Could you please decribe for what purpose it will be used?
> >>
> >> I mean why one might have to either exclude or run only regression
> tests?
> >
> >If running all tests takes up much time, running all regression test
> >(for one particular version) may be a convenient way to verify the
> >correctness of bug-fixing.
>
> I can be used for informational purposes as well. If the test fails, you
> need simply re-open the associated bug provided the bug number can be
> easily found.
>
> Regards,
> Alexey.
>
> >
> >Best regards,
> >Richard.
> >
> >>
> >> Thanks,
> >> Mikhail
> >>
> >> 2006/8/30, Vladimir Ivanov <ivavladimir@gmail.com>:
> >>> On 8/30/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> >>> >
> >>> >
> >>> >
> >>> > Vladimir Ivanov wrote:
> >>> > > Also some tag for regression tests should be added.
> >>> > Yes. Do you think we could annotate regression test as
> >>> > *level.regression*? Thanks a lot.
> >>>
> >>>
> >>> Yes, I do. While tests can have more than one group it will enough.
> >>>  thanks, Vladimir
> >>>
> >>>
> >>> Richard
> >>> > > thanks, Vladimir
> >>> > >
> >>> > >
> >>> > > On 8/28/06, Richard Liang <richard.liangyx@gmail.com> wrote:
> >>> > >>
> >>> > >>
> >>> > >>
> >>> > >> Richard Liang wrote:
> >>> > >> > Hello All,
> >>> > >> >
> >>> > >> > Now let's talk about the TestNG groups. I have read the
> related
> >>> > >> > threads which posted by George, Vladimir Ivanov and Alexei
> >>> Zakharov.
> >>> > >> > All of them are good discussion about TestNG groups.
> >>> > >> >
> >>> > >> > IMHO, we may define Harmony test groups according the
> following 4
> >>> > >> > dimensions:
> >>> > >> >
> >>> > >> > 1) [Platform] os.any, os.<platform id>
> >>> > >> > *os.any* - group of tests which pass on any platform.
IMHO,
> >>> most of
> >>> > >> > our tests should be in this group.
> >>> > >> > *os.<platform id>* - group of tests which are designed
for
> one
> >>> > >> > specific platform. A test may be in more than one of
the
> >>> groups. e.g
> >>> > .,
> >>> > >> > @Test(groups={"os.win.IA32", "os.linux.IA32"})
> >>> > >> >
> >>> > >> >    ** os.any and os.<platform id> are mutually
exclusive,
> that
> >>> is,
> >>> > >> > tests in os.any group should not be in os.win.IA32.
> >>> > >> >
> >>> > >> > 2) [Test state] state.broken, state.broken.<platform
id>
> >>> > >> > *state.broken* - group of tests which fail on every platform,
> >>> because
> >>> > >> > of bugs of tests or implementation. We need to fix the
bugs
> of
> >>> tests
> >>> > >> > or implementation to make them pass.
> >>> > >> > *state.broken.<platform id>* - groups of test which
only fail
> >>> on one
> >>> > >> > specific platform. A test may be in more than one of
the
> >>> groups. e.g
> >>> > .,
> >>> > >> > @Test(groups={"state.broken.linux.IA32",
> "os.broken.linux.IA64"})
> >>> > >> >
> >>> > >> >     **state.broken.<platform id> group may be used
as a
> >>> convenient
> >>> > way
> >>> > >> > to indicate that a test is platform-specific. e.g., If
we
> >>> support 10
> >>> > >> > platforms, and one test are designed for 9 platforms
except
> for
> >>> > MacOS,
> >>> > >> > instead of list 9 os.<platform id>, we can just
use
> >>> > state.broken.MacOS
> >>> > >> >
> >>> > >> > 3) [Test type] type.api , type.impl
> >>> > >> > *type.api* - group of tests which are tests for APIs
in the
> Java
> >>> > >> > Specification
> >>> > >> > *type.impl* - groups of tests which are tests for
> >>> Harmony-specific
> >>> > >> > implementation
> >>> > >> >
> >>> > >> >     ** type.api and type.impl are also mutually exclusive.
> >>> > >> >
> >>> > >> > 4) [Test Level] level.unit, level.integration, level.system,
> >>> > >> > level.stress, etc. (Levels of Test refer to the increase
in
> >>> > complexity
> >>> > >> > as moving through test cycle. )
> >>> > >> >    ** A test may be in more than one of the groups.
> >>> > >> >    ** In fact, some tests such as System tests are the
> >>> verification
> >>> > of
> >>> > >> > the entire system.  Maybe we'll put them into a separate
> project.
> >>> > >> > e.g., harmony/enhanced/SVT (System Verification Test).
> >>> > >> >
> >>> > >> > If we want to run all the unit test for APIs on windows,
we
> >>> may use
> >>> > >> > TestNG groups to select the tests:
> >>> > >> >        <groups>
> >>> > >> >            <run>
> >>> > >> >                <include name="os.any" />
> >>> > >> >                <include name="type.api" />
> >>> > >> >                <include name="os.win.IA32" />
> >>> > >> >                <exclude name=" state.broken" />
> >>> > >> >                <exclude name="state.broken.win.IA32"
/>
> >>> > >> >            </run>
> >>> > >> >        </groups>
> >>> > >> >
> >>> > >> Hello All,
> >>> > >>
> >>> > >> I'm sorry. It seems that the example does not work. I will
try
> to
> >>> > figure
> >>> > >> another example soon. ;-)
> >>> > >>
> >>> > >> Best regards,
> >>> > >> Richard
> >>> > >> >
> >>> > >> > Well, I think our most of existing tests are in the groups
of
> >>> > >> > {"os.any", "type.api", "level.unit"}, and I have asked
TestNG
> >>> to add
> >>> > a
> >>> > >> > new option "-groups" for its JUnitConverter which allow
us to
> >>> specify
> >>> > >> > the test groups when migrate from JUnit test to TestNG
test.
> >>> > >> >
> >>> > >> > Thanks for reading so far, and I will highly appreciate
your
> >>> comments
> >>> > >> > or suggestion.  ;-)
> >>> > >> >
> >>> > >>
> >>> > >> --
> >>> > >> Richard Liang
> >>> > >> China Software Development Lab, IBM
> >>> > >>
> >>> > >>
> >>> > >>
> >>> > >>
> >>>
> ---------------------------------------------------------------------
> >>> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> > >> To unsubscribe, e-mail:
> >>> harmony-dev-unsubscribe@incubator.apache.org
> >>> > >> For additional commands, e-mail:
> >>> harmony-dev-help@incubator.apache.org
> >>> > >>
> >>> > >>
> >>> > >
> >>> >
> >>> > --
> >>> > Richard Liang
> >>> > China Software Development Lab, IBM
> >>> >
> >>> >
> >>> >
> >>> >
> ---------------------------------------------------------------------
> >>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> >>> > To unsubscribe, e-mail:
> harmony-dev-unsubscribe@incubator.apache.org
> >>> > For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> >>> >
> >>> >
> >>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail:
> harmony-dev-help@incubator.apache.org
> >>
> >>
> >
> >--
> >Richard Liang
> >China Software Development Lab, IBM
> >
> >
> >
> >---------------------------------------------------------------------
> >Terms of use : http://incubator.apache.org/harmony/mailing.html
> >To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> --
> Alexey A. Ivanov
> Intel Middleware Product Division
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message