openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pinaki Poddar (JIRA)" <>
Subject [jira] Commented: (OPENJPA-949) Allow override of Surefire test excludes from cmdline
Date Mon, 02 Mar 2009 21:28:56 GMT


Pinaki Poddar commented on OPENJPA-949:

Previous discussion on filtering tests, or allowing the execution to ignore tests that fail
-- we agreed upon few broad goals:
1. The information/specification about a test being excluded or allowed to fail should be
as close as possible to the test itself. We basically went in a direction opposite to having
a excluded list of tests in a separate file. 
2. That earlier discussion gave rise to devising @AllowFailure annotation that one can attach
to a Test itself to control behavior when the test fails. The test harness was upgraded to
process the @AllowFailure annotation.

Given its recent usage, I believe that the developers find the mechanics useful.

In view of that, I did have reservation about reintroducing an exclude list for tests. But
I did suggest an exclusion list. So let me first elaborate the purpose or intended usage behind
raising the current issue.

1. Current OpenJPA test corpus takes roughly 40 minutes to run on a TeamCity server. 
2. Often a developer has an estimate of 'region of impact' a fix/change is going to make.
For example, run all the tests of such and such package.
3. The developer wants to submit a TeamCity run on that estimated 'region of impact'. The
submission is not for commit -- but to get an initial confirmation of a suggested fix or change.

4. Currently the confirmation takes at the worst 40 minutes and at best much quicker if happens
to fail a test that appears earlier in the predetermined sequence TeamCity executes tests.
5. The current worst case wait for 40 minutes can be reduced, if we get a control to tell
TeamCity which subsets of tests to execute.

The challenge of devising a mechanics is how to enforce or, at least, minimize the risk that
fix/changes do not get committed without covering the full corpus of tests.

One of the obvious places to specify this filtering is pom.xml as Micheal Dick has suggested.
The suggestion in the submitted patch based on Geronimo practices and using Groovy also seems
to be along the similar line. 
It is bit risky in my view, if the filtered test execution gets inadvertently committed. Then
the committed change in pom.xml will impact everyone else's test execution. Having a separate,
constant all-inclusive filter file which gets overwritten at the beginning by the execution
owner's filter but again restored to its pristine state at the end of the execution (controlled
by maven build phases) will minimize such risks of inadvertent commits to other developers.
Also personally I find it easier to edit a simple text file rather than pom.xml.



> Allow override of Surefire test excludes from cmdline
> -----------------------------------------------------
>                 Key: OPENJPA-949
>                 URL:
>             Project: OpenJPA
>          Issue Type: Improvement
>          Components: build / infrastructure
>            Reporter: Donald Woods
>            Assignee: Donald Woods
>            Priority: Minor
>             Fix For: 2.0.0
>         Attachments: OPENJPA-949-trunk-20090302.patch
> Allow users to provide their own Surefire excludes list through an external file.
> This can be accomplished by using the gmaven-plugin and some Groovy code.
> The default set of excludes will still be provided and used by default in the builds,
but will be moved to src/test/resources/excludes.xml.
> Users can provide their own version of excludes file by:
> mvn -DsurefireExcludesFile=<filename>
> where <filename> could be a local file like:
> -DsurefireExcludesFile=/tmp/excludes.xml
> or from a network share like:
> -DsurefireExcludesFile=/mnt/public/openjpa-persistence-jdbc/excludes.xml

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message