maven-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arnaud HERITIER" <aherit...@gmail.com>
Subject Re: Sane plugin testing
Date Thu, 07 Aug 2008 20:43:50 GMT
As I said in another thread, we just ended 3.
We named the profile integration-tests but to have the same name as in the
core we'll rename it run-its

It seems that there is 2 point of view about ITs activation :
-1) ITs are part of the build and always launched
-2) ITs are launched on demand (because they are consumming a lot of time
and ressources)
What do you prefer ?
For me it's (2).

Another question : What is an integration test ?
For me unit tests don't handle resources like files. In unit tests we are
using mocks to be able to use only in-memory tests.
Everything else which is reading poms, settings or other resources are
integration tests (whatever the framework used : maven-embedder,
maven-verifier, ...).
The problem actually is that our APIs aren't often developped to have unit
tests.
For example, we often have in parameters InputStream or something like than
instead of a Project object which can be mocked.
We should have to review them to be able to have many more UTs.
Like you can see in [1] we don't have an important coverage of UTs.

ITs are needed for example in plugins when we wrap another tool, but
otherwise, and particularly in the core we should have a good unit tests
suite.

[1] http://nemo.sonar.codehaus.org/project/dashboard/19084?sid=669108


Arnaud

On Wed, Aug 6, 2008 at 5:24 PM, Jason van Zyl <jason@maven.org> wrote:

> Hi,
>
> I think we've gotten to the point where we need to decide how we are going
> to test plugins. We need to pick one of the frameworks, settle on a pattern,
> and use that in the plugins otherwise there will be no sane way to validate
> a set of plugins works against a given version of Maven. What I'm thinking
> about here concretely is testing all the plugins that we have here against
> Maven 2.1 to know that we have not screwed something up so terribly that
> things like the deploy plugin doesn't work, or whatever.
>
> I think how this starts is that we:
>
> 1) Pick one of the tools
> 2) Create a touchstone project that can be expanded where necessary for any
> given plugin so that we have a baseline project against which to test
> 3) Pick a standard profile name for invoking this test
>
> This way we create a standard hook point for a larger harness to get hold
> off. We can check out sources and create an aggregator POM with the given
> profile activated to test a set of plugins. I don't know yet what the best
> way would be to share a touchstone project (and that is not  to say we won't
> need different projects but we have to start with a baseline), but once we
> start this we can also start plugging in other things like integration
> testing that includes things like coverage or whatever else.
>
> I think the key in moving forward is getting 1-3 sorted out so we're not
> using 5 frameworks and testing plugins with N different patterns where it's
> impossible to hook into for larger scale testing. I think this is the only
> way forward to validate that a set of plugins work against a given version
> of Maven which is vital information to know before releasing 2.1.
>
> For integration testing I have found the SHITTY plugin (we would simply
> have to change that name, sorry Jason Dillon) to be the most useful and
> feature rich. Should be relatively simple to create a test project, and a
> profile name (run-its like the core ITs). Then we figure out how to share
> and version the test project to create a stable baseline. I chatted about
> this briefly in IRC with Benjamin and wanted to get the information out. I
> think it's vital to get this rolling if we want to roll out a 2.1-alpha-1
> with some degree of confidence we have toasted a bunch of plugins due to
> incompatibilities in the core.
>
> Thanks,
>
> Jason
>
> ----------------------------------------------------------
> Jason van Zyl
> Founder,  Apache Maven
> jason at sonatype dot com
> ----------------------------------------------------------
>
> What matters is not ideas, but the people who have them. Good people can
> fix bad ideas, but good ideas can't save bad people.
>
>  -- Paul Graham
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>


-- 
..........................................................
Arnaud HERITIER
..........................................................
OCTO Technology - aheritier AT octo DOT com
www.octo.com | blog.octo.com
..........................................................
ASF - aheritier AT apache DOT org
www.apache.org | maven.apache.org
...........................................................

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message