From dev-return-39032-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org Wed Mar 10 02:51:42 2010 Return-Path: Delivered-To: apmail-harmony-dev-archive@www.apache.org Received: (qmail 90298 invoked from network); 10 Mar 2010 02:51:42 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Mar 2010 02:51:42 -0000 Received: (qmail 81431 invoked by uid 500); 10 Mar 2010 02:51:13 -0000 Delivered-To: apmail-harmony-dev-archive@harmony.apache.org Received: (qmail 81243 invoked by uid 500); 10 Mar 2010 02:51:12 -0000 Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list dev@harmony.apache.org Received: (qmail 81234 invoked by uid 99); 10 Mar 2010 02:51:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Mar 2010 02:51:12 +0000 X-ASF-Spam-Status: No, hits=4.4 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of littlee1032@gmail.com designates 209.85.223.175 as permitted sender) Received: from [209.85.223.175] (HELO mail-iw0-f175.google.com) (209.85.223.175) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Mar 2010 02:51:11 +0000 Received: by iwn5 with SMTP id 5so2133315iwn.24 for ; Tue, 09 Mar 2010 18:50:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=j4hAEua1ZBd/Rzn8vJNgan63m5x0jaUm6RpJ6v4Y49k=; b=FjBntO2NWnlC9LHqxdmuWQbeXqFdv22UwP8RbopDzkSqrWqK32Bday8juyvct3tCj1 m+qYTNGvbCxPDoEIVEcMykgR4s09sTPSKaoeBngdRC0OtGcGgzc9uBnf7ASqPKwu01bP 5ueJiCDPTmAdCG8qsPsrPwN4R8NcYW7NOZEoQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=t2vjrj2CAd3TqPXKh7FCtVZiZW5wOhqffN9XOSq8dz4jlPIn4PRYBCuwOLwlUnH3Ye Udu8ES1LCFDM0rqZEdTfTq8Y9daZ3hrkCAq65BRpl9zNzjP7H+xskD1nPjGFsBfWXkKY Z7sBLkKoUGZXUyBghZpCrE+yufILDQMgmqC+A= MIME-Version: 1.0 Received: by 10.231.151.212 with SMTP id d20mr302277ibw.53.1268189450210; Tue, 09 Mar 2010 18:50:50 -0800 (PST) In-Reply-To: <3b3f27c61003091701t6b73cf8ev3eb23779ce876dc9@mail.gmail.com> References: <3b3f27c61003091701t6b73cf8ev3eb23779ce876dc9@mail.gmail.com> Date: Wed, 10 Mar 2010 10:50:50 +0800 Message-ID: <5948b71e1003091850h58dcf58dq37bcdc77bbacb747@mail.gmail.com> Subject: Re: Rethinking testing conventions From: Charles Lee To: dev@harmony.apache.org Content-Type: multipart/alternative; boundary=0016e68ddac7b010160481695d86 --0016e68ddac7b010160481695d86 Content-Type: text/plain; charset=ISO-8859-1 That's great. But most old test case is in JUnit 3.8 style. We can follow the latest junit when we add the new test cases. What about the old ones? On Wed, Mar 10, 2010 at 9:01 AM, Nathan Beyer wrote: > I've been experimenting with JUnit4 extensions in an attempt to find a > more simple approach to our basic testing needs. The two major > requirements that I see in Harmony's test infrastructure (at least the > classlib stuff) are - exclusions and platform-specific tests. (I'm > ignoring the bootstrap-classpath stuff for now.) To handle these > today, we have exclusion files, which are loaded and used to exclude > certain files and then we branch the test folders by platform. Each > of these has issues - exclusions are skipping whole files, not just > specific failing tests; exclusions aren't well reported on; test files > in multiple branches can have duplicate code, etc. > > One concept I've been working with is using annotations to describe > the tests for the purposes of exclusions and for platform definition. > The annotations can then be utilized in many ways via JUnit - method > rules, request processing filters and others. Here's an example of how > the tests might look. > > class FileTest { > @Test > @Platform(os="windows") > public void testSomethingOnWindows { } > > @Test > @Platform(os="linux", arch="x86_64") > public void testSomethingOnLinuxX86_64 {} > > @Test > @Exclude(os="windows",vm="drlvm") > public void testSomethingExcludedOnWindowsDRLVM {} > } > > Thoughts? Comments? > -- Yours sincerely, Charles Lee --0016e68ddac7b010160481695d86--