Return-Path: 
 <harmony-dev-return-12556-apmail-incubator-harmony-dev-archive=incubator.apache.org@incubator.apache.org>
Delivered-To: apmail-incubator-harmony-dev-archive@www.apache.org
Received: (qmail 18782 invoked from network); 28 Aug 2006 08:23:00 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199)
  by minotaur.apache.org with SMTP; 28 Aug 2006 08:23:00 -0000
Received: (qmail 64046 invoked by uid 500); 28 Aug 2006 08:22:58 -0000
Delivered-To: apmail-incubator-harmony-dev-archive@incubator.apache.org
Received: (qmail 63773 invoked by uid 500); 28 Aug 2006 08:22:57 -0000
Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:harmony-dev-help@incubator.apache.org>
List-Unsubscribe: <mailto:harmony-dev-unsubscribe@incubator.apache.org>
List-Post: <mailto:harmony-dev@incubator.apache.org>
List-Id: <harmony-dev.incubator.apache.org>
Reply-To: harmony-dev@incubator.apache.org
Delivered-To: mailing list harmony-dev@incubator.apache.org
Received: (qmail 63762 invoked by uid 99); 28 Aug 2006 08:22:57 -0000
Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Aug 2006 01:22:57 -0700
X-ASF-Spam-Status: No, hits=0.0 required=10.0
	tests=
X-Spam-Check-By: apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Received: from [192.55.52.88] (HELO fmsmga101-1.fm.intel.com) (192.55.52.88)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Aug 2006 01:22:56 -0700
Received: from fmsmga001.fm.intel.com ([10.253.24.23])
  by fmsmga101-1.fm.intel.com with ESMTP; 28 Aug 2006 01:22:36 -0700
Received: from fmsmsx334.fm.intel.com (HELO fmsmsx334.amr.corp.intel.com)
 ([132.233.42.1])
  by fmsmga001.fm.intel.com with ESMTP; 28 Aug 2006 01:22:35 -0700
X-ExtLoop1: 1
X-IronPort-AV: i="4.08,175,1154934000";
   d="scan'208"; a="122096260:sNHT78433362"
Received: from fmsmsx312.amr.corp.intel.com ([132.233.42.227]) by
 fmsmsx334.amr.corp.intel.com with Microsoft SMTPSVC(6.0.3790.1830);
	 Mon, 28 Aug 2006 01:22:29 -0700
Received: from mssmsx411.ccr.corp.intel.com ([10.125.2.10]) by
 fmsmsx312.amr.corp.intel.com with Microsoft SMTPSVC(6.0.3790.1830);
	 Mon, 28 Aug 2006 01:22:10 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: [classlib][TestNG] groups of Harmony test
Date: Mon, 28 Aug 2006 12:22:06 +0400
Message-ID: <8E389A5F2FEABA4CB1DEC35A25CB39CE30F42E@mssmsx411>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: [classlib][TestNG] groups of Harmony test
Thread-Index: AcbIWcmO59kwFKF7RWSsLz1SovpaIQCILvXA
From: "Ivanov, Alexey A" <alexey.a.ivanov@intel.com>
To: <harmony-dev@incubator.apache.org>
X-OriginalArrivalTime: 28 Aug 2006 08:22:10.0047 (UTC)
 FILETIME=[0E8BD0F0:01C6CA7B]
X-Virus-Checked: Checked by ClamAV on apache.org
X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N


>-----Original Message-----
>From: Richard Liang [mailto:richard.liangyx@gmail.com]
>Sent: Friday, August 25, 2006 7:18 PM
>To: harmony-dev@incubator.apache.org
>Subject: [classlib][TestNG] groups of Harmony test
>
>Hello All,
>
>Now let's talk about the TestNG groups. I have read the related threads
>which posted by George, Vladimir Ivanov and Alexei Zakharov. All of
them
>are good discussion about TestNG groups.
>
>IMHO, we may define Harmony test groups according the following 4
>dimensions:
>
>1) [Platform] os.any, os.<platform id>
> *os.any* - group of tests which pass on any platform. IMHO, most of
our
>tests should be in this group.
> *os.<platform id>* - group of tests which are designed for one
specific
>platform. A test may be in more than one of the groups. e.g.,
>@Test(groups=3D{"os.win.IA32", "os.linux.IA32"})
>
>    ** os.any and os.<platform id> are mutually exclusive, that is,
>tests in os.any group should not be in os.win.IA32.
>
>2) [Test state] state.broken, state.broken.<platform id>
> *state.broken* - group of tests which fail on every platform, because
>of bugs of tests or implementation. We need to fix the bugs of tests or
>implementation to make them pass.
> *state.broken.<platform id>* - groups of test which only fail on one
>specific platform. A test may be in more than one of the groups. e.g.,
>@Test(groups=3D{"state.broken.linux.IA32", "os.broken.linux.IA64"})
>
>     **state.broken.<platform id> group may be used as a convenient way
>to indicate that a test is platform-specific. e.g., If we support 10
>platforms, and one test are designed for 9 platforms except for MacOS,
>instead of list 9 os.<platform id>, we can just use state.broken.MacOS
>
>3) [Test type] type.api, type.impl
> *type.api* - group of tests which are tests for APIs in the Java
>Specification
> *type.impl* - groups of tests which are tests for Harmony-specific
>implementation
>
>     ** type.api and type.impl are also mutually exclusive.
>
>4) [Test Level] level.unit, level.integration, level.system,
>level.stress, etc. (Levels of Test refer to the increase in complexity
>as moving through test cycle. )
>    ** A test may be in more than one of the groups.
>    ** In fact, some tests such as System tests are the verification of
>the entire system.  Maybe we'll put them into a separate project. e.g.,
>harmony/enhanced/SVT (System Verification Test).

5) [Environment] env.display, env.headless
   To distinguish AWT and Swing tests which need a display to run, and
those which don't, as Mark proposed [1].

Regards,
Alexey.

[1]
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200608.mb
ox/%3c200608241441.k7OEfuBR018998@d06av02.portsmouth.uk.ibm.com%3e

>
>If we want to run all the unit test for APIs on windows, we may use
>TestNG groups to select the tests:
>        <groups>
>            <run>
>                <include name=3D"os.any" />
>                <include name=3D"type.api" />
>                <include name=3D"os.win.IA32" />
>                <exclude name=3D"state.broken" />
>                <exclude name=3D"state.broken.win.IA32" />
>            </run>
>        </groups>
>
>
>Well, I think our most of existing tests are in the groups of
{"os.any",
>"type.api", "level.unit"}, and I have asked TestNG to add a new option
>"-groups" for its JUnitConverter which allow us to specify the test
>groups when migrate from JUnit test to TestNG test.
>
>Thanks for reading so far, and I will highly appreciate your comments
or
>suggestion.  ;-)
>
>--
>Richard Liang
>China Software Development Lab, IBM
>
>
>
>---------------------------------------------------------------------
>Terms of use : http://incubator.apache.org/harmony/mailing.html
>To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>For additional commands, e-mail: harmony-dev-help@incubator.apache.org

--
Alexey A. Ivanov
Intel Middleware Product Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org