deltaspike-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Struberg <>
Subject Re: basic decisions: test setup
Date Sat, 17 Dec 2011 10:05:11 GMT
Hi Christian!

@TargetContainer sounds good, but I'd rather make it a 


with values: SE, Servlet, EE-light, EE-full

The excludes should be done in the single test integration modules, and the goal should be
to get down to zero excluded tests for all containers.

@Jason: that mixture for the tests looks fine to me as well. Can you please hack a sample
structure in the DeltaSpikePlayground git repo?


> From: Christian Kaltepoth <>
>Sent: Saturday, December 17, 2011 7:55 AM
>Subject: Re: basic decisions: test setup
>I would like to add some "non-binding" comments here regarding 3).
>Actually I really like the concept Jason described and which is
>currently used in Seam3. Having worked with it a bit I can confirm
>that it runs fine. And somehow it's the best of both worlds (in regard
>to 3a vs 3b).
>That the full integration test suite with all containers cannot run in
>a single Maven invocation is IMHO not a real problem. We could write a
>simple script for this.
>The only thing that could be optimized in this setup is the separation
>of container dependent tests via package name. But perhaps this could
>be solved by Arquillian? What I would really like to see is the
>ability to configure a test so it is only executed in a specific
>container (something similar to @TargetsContainer). This way tests
>that require to be run in a full container could be executed only in
>JBoss and Glassfish but would be skipped in embedded Weld.
>2011/12/16 Jason Porter <>:
>> Comments inline.
>> On Fri, Dec 16, 2011 at 01:38, Mark Struberg <> wrote:
>>> Hi!
>>> Another round of discussion - this time about how to setup our unit
>>> testing and integration testing.
>>> I think it's pretty much clear that we will gonna use Arquillian for all
>>> our tests. That was also fundamental part of the incubator proposal.
>>> And instead of inventing yet another abstraction layer, I'd favour to just
>>> use Arquillian and contribute all things we need to this project.
>>> So here a formal vote
>>> 1.)
>>> for using Arquillian as test integration framework.
>>>  +1 from me
>> +1 from me
>>> 2.)
>>> What do we like to use from the unit tests itself? TestNG or JUnit?
>>> JUnit has better Arquillian integration and TestNG is better for 'real
>>> world' projects, because it allows to define test dependencies.
>>> So which one to take?
>>> But we should definitely only use 1 of the two exclusively!
>> +1 for using one exclusively
>> As for which, I'm also +0 I like both for different reasons, but JUnit is
>> certainly the easier of the two to use with Arquillian.
>> Also, are we open to having other languages for tests or using other
>> testing structures such as spock (it works with Arquillian, BTW), RSpec,
>> scala, etc.?
>>> 3.)
>>> 'Integrated' tests vs 'Integration Tests'
>>> We have 2 options to test our projects
>>> 3.a.) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add a profile for each and every server (maybe we can trim this down
>>> with pom imports?).
>>> Then run the build with one after each other:
>>> $> mvn clean test -Powb
>>> $> mvn clean test -Pweld
>>> $> mvn clean test -Powb-tc
>>> $> mvn clean test -Pweld-tc
>>> $> mvn clean test -Pgf31
>>> $> mvn clean test -Pas7
>>> $> mvn clean test -Ptomee
>>> This might pretty much blow up our poms...
>>> 3.b) create a full unit test in each module (e.g. deltaspike/core/impl)
>>> and add only 2 profiles directly ('weld' and 'owb' (probably resin later))
>>> Then add a deltaspike/test/base/core with the very basic parent stuff and
>>> 1 module for each integrated container, e.g.
>>> deltaspike/test/weld-tc
>>> deltaspike/test/owb-tc
>>> deltaspike/test/weld-jetty
>>> deltaspike/test/owb-jetty
>>> deltaspike/test/tomee
>>> deltaspike/test/gf31
>>> deltaspike/test/geronimo
>>> deltaspike/test/websphere
>>> deltaspike/test/as7
>>> deltaspike/test/as6
>>> etc.
>>> The initial setup costs are higher, but it would be pretty easy to add new
>>> containers that way.
>>> Which route shall we take?
>> In Seam 3 we have an approach which is a little bit of a mixture of both.
>> We have regular unit tests (with mockito, stubs, etc) under
>> impl/src/test/... and then we have a separate module called testsuite which
>> contained all the integration tests. We used pom imports to get all the
>> containers setup correctly and then had different profiles which pulled in
>> the poms and also configured surefire for that server. Inside the code
>> structure we separated the servers by package: common, jbossas, glassfish,
>> etc. That way all of the tests were in one location, they were easy to
>> execute, the tests stayed DRY, and the IDE wasn't much of a problem. The
>> two downsides we had were 1) the extra config in the pom and 2) you can't
>> run the full integration suite in one maven execution. We spent quite a bit
>> of time examining options and trying some of them out. This was the best we
>> came up with, and also the one Aslak thought would work best as well.
>>> LieGrue,
>>> strub
>> --
>> Jason Porter
>> Software Engineer
>> Open Source Advocate
>> Author of Seam Catch - Next Generation Java Exception Handling
>> PGP key id: 926CCFF5
>> PGP key available at:,
>Christian Kaltepoth

View raw message