harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Beyer <ndbe...@apache.org>
Subject Re: Rethinking testing conventions
Date Thu, 11 Mar 2010 04:09:44 GMT
On Wed, Mar 10, 2010 at 7:38 PM, Charles Lee <littlee1032@gmail.com> wrote:
> On Wed, Mar 10, 2010 at 9:01 AM, Nathan Beyer <ndbeyer@apache.org> wrote:
>> I've been experimenting with JUnit4 extensions in an attempt to find a
>> more simple approach to our basic testing needs. The two major
>> requirements that I see in Harmony's test infrastructure (at least the
>> classlib stuff) are - exclusions and platform-specific tests. (I'm
>> ignoring the bootstrap-classpath stuff for now.) To handle these
>> today, we have exclusion files, which are loaded and used to exclude
>> certain files and then we branch the test folders by  platform. Each
>> of these has issues - exclusions are skipping whole files, not just
>> specific failing tests; exclusions aren't well reported on; test files
>> in multiple branches can have duplicate code, etc.
> We have exclusion file mostly because it has test case fail. It seems some
> one put the test case to the exclusion list but forget to put it out. And
> after a long time, no one would remember this. In this situation, I'd like
> to use @Ignore("reason") annotation, not simply exclude all the test cases
> in this file, but ignore the specific test case with reasons.

We can use @Ignore, if we want it to be excluded for all runs. I've
been playing around with a custom annotation, the '@Exclude', that is
essentially the same thing, but adds 'os', 'arch' and 'vm' properties
to it. We could do this with the existing @Ignore and just put an
expression prefix in the text of the annotation.

>> One concept I've been working with is using annotations to describe
>> the tests for the purposes of exclusions and for platform definition.
>> The annotations can then be utilized in many ways via JUnit - method
>> rules, request processing filters and others. Here's an example of how
>> the tests might look.
>> class FileTest {
>>  @Test
>>  @Platform(os="windows")
>>  public void testSomethingOnWindows { }
>>  @Test
>>  @Platform(os="linux", arch="x86_64")
>>  public void testSomethingOnLinuxX86_64 {}
>>  @Test
>>  @Exclude(os="windows",vm="drlvm")
>>  public void testSomethingExcludedOnWindowsDRLVM {}
>> }
>> Thoughts? Comments?
> --
> Yours sincerely,
> Charles Lee

View raw message