harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [classlib] do we need a global exclude list?
Date Thu, 16 Feb 2006 09:31:59 GMT
Geir Magnusson Jr wrote:
> Stepan Mishura wrote:
>> Did anybody think of creating a 'global' (i.e. shared by all modules)
>> exclude list or every module will have its own exclude list? Or Harmony
>> tests will always pass and we don't need it at all :-)
> 
> That would be the goal :)
> 
>> I see at least the following benefits of creating 'global' exclude
>> list: all
>> know issues are kept in one well known place (they don't spread between
>> several private lists)
> 
> That's true, but....  I always imagined that people would be working in
> the modules anyway, so there isn't much gain.

Modules should be as self-sufficient as we can make them, so devolving
tests and their exclude lists into a module makes sense to me too.  They
can be linked into a global view quite easily (read on...)

> also it is easier to create an exclude list for a
>> target platform, for example, linux.exclude or win.exclude.
> 
> Yes, that could be.  Interesting idea.

The mechanism that George contributed in HARMONY-57 uses a decorator to
implement an exclusion list on regular JUnit tests.  (I happen to know
that George is off enjoying himself for the next few days, so I hope he
doesn't mind me describing it here.)

The exclusion list is implemented as a (declarative) XML file read when
the tests run -- in HARMONY-57 you can see one in
Harmony_Tests/src/test/resources/config/jcltest-excludes.xml

Picking an entry at random from there:
    ...
    <hy:type id="tests.api.java.io.FileTest">
        <hy:exclude id="test_Constructor_String_String_112270"
                    shouldfix="true">
            <hy:reason>Undiagnosed failure</hy:reason>
        </hy:exclude>
        <hy:exclude id="test_Constructor_File_String_112270"
                    shouldfix="true">
            <hy:reason>Undiagnosed failure</hy:reason>
        </hy:exclude>
        <hy:exclude id="test_deleteOnExit" shouldfix="true">
            <hy:reason>Needs investigation...</hy:reason>
        </hy:exclude>
    </hy:type>
    ...

You can see how it works -- some tests are excluded because they fail
and should be fixed, others (not shown here) can be excluded because
they don't make sense on a particular platform, VM, etc.

Applying a style sheet makes it easy to read the exclusions list in
glorious technicolor, either in an individual module or as a combined
global view.

Take a look at the incoming contribution and see if it fits your needs.

Regards,
Tim

> However, I did imagine that we'd give the modules a bit of freedom and
> independence for testing - a global exclude list might impact that?
> 
> geir
> 
>>
>> Thoughts?
>>
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

Mime
View raw message