deltaspike-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Struberg <strub...@yahoo.de>
Subject Re: basic decisions: test setup
Date Sat, 17 Dec 2011 15:56:57 GMT
We might solve this simply with a package naming rule. And then we configure surefire to just
run the tests with the respective package?

Not the most advanced solution, but will work.


LieGrue,
strub



----- Original Message -----
> From: Christian Kaltepoth <christian@kaltepoth.de>
> To: deltaspike-dev@incubator.apache.org
> Cc: 
> Sent: Saturday, December 17, 2011 4:43 PM
> Subject: Re: basic decisions: test setup
> 
> @Mark:
> 
> Actually @TargetsContainer is an existing feature of Arquillian. It
> allows to build container specific deployments.
> 
> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
> 
> Unfortunately it won't fit our needs because the test completely fails
> if there is no matching deployment for the container the test is
> executed against. So I guess we have to build something new for this.
> 
> I really like your suggestion regarding the four environments. It
> absolutely make sense to declare environment constraints (like SE,
> Servlet, EE, etc.) instead of listing concrete containers.
> 
> Christian
> 
> 
> 
> 2011/12/17 Mark Struberg <struberg@yahoo.de>:
>>  Hi Christian!
>> 
>> 
>>  @TargetContainer sounds good, but I'd rather make it a
>> 
>> 
>>  @TargetEnvironment()
>> 
>>  with values: SE, Servlet, EE-light, EE-full
>> 
>>  The excludes should be done in the single test integration modules, and the 
> goal should be to get down to zero excluded tests for all containers.
>> 
>> 
>>  @Jason: that mixture for the tests looks fine to me as well. Can you please 
> hack a sample structure in the DeltaSpikePlayground git repo?
>> 
>> 
>>  LieGrue,
>>  strub
>> 
>> 
>> 
>> 
>>> ________________________________
>>>  From: Christian Kaltepoth <christian@kaltepoth.de>
>>> To: deltaspike-dev@incubator.apache.org
>>> Sent: Saturday, December 17, 2011 7:55 AM
>>> Subject: Re: basic decisions: test setup
>>> 
>>> I would like to add some "non-binding" comments here regarding 
> 3).
>>> 
>>> Actually I really like the concept Jason described and which is
>>> currently used in Seam3. Having worked with it a bit I can confirm
>>> that it runs fine. And somehow it's the best of both worlds (in 
> regard
>>> to 3a vs 3b).
>>> 
>>> That the full integration test suite with all containers cannot run in
>>> a single Maven invocation is IMHO not a real problem. We could write a
>>> simple script for this.
>>> 
>>> The only thing that could be optimized in this setup is the separation
>>> of container dependent tests via package name. But perhaps this could
>>> be solved by Arquillian? What I would really like to see is the
>>> ability to configure a test so it is only executed in a specific
>>> container (something similar to @TargetsContainer). This way tests
>>> that require to be run in a full container could be executed only in
>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>> 
>>> Christian
>>> 
>>> 
>>> 2011/12/16 Jason Porter <lightguard.jp@gmail.com>:
>>>>  Comments inline.
>>>> 
>>>>  On Fri, Dec 16, 2011 at 01:38, Mark Struberg 
> <struberg@yahoo.de> wrote:
>>>> 
>>>>>  Hi!
>>>>> 
>>>>>  Another round of discussion - this time about how to setup our 
> unit
>>>>>  testing and integration testing.
>>>>> 
>>>>>  I think it's pretty much clear that we will gonna use 
> Arquillian for all
>>>>>  our tests. That was also fundamental part of the incubator 
> proposal.
>>>>>  And instead of inventing yet another abstraction layer, I'd 
> favour to just
>>>>>  use Arquillian and contribute all things we need to this 
> project.
>>>>>  So here a formal vote
>>>>> 
>>>>>  1.)
>>>>> 
>>>>>  for using Arquillian as test integration framework.
>>>>>   +1 from me
>>>>> 
>>>> 
>>>>  +1 from me
>>>> 
>>>> 
>>>>> 
>>>>>  2.)
>>>>> 
>>>>>  What do we like to use from the unit tests itself? TestNG or 
> JUnit?
>>>>>  JUnit has better Arquillian integration and TestNG is better 
> for 'real
>>>>>  world' projects, because it allows to define test 
> dependencies.
>>>>>  So which one to take?
>>>>>  But we should definitely only use 1 of the two exclusively!
>>>>> 
>>>> 
>>>>  +1 for using one exclusively
>>>> 
>>>>  As for which, I'm also +0 I like both for different reasons, 
> but JUnit is
>>>>  certainly the easier of the two to use with Arquillian.
>>>> 
>>>>  Also, are we open to having other languages for tests or using 
> other
>>>>  testing structures such as spock (it works with Arquillian, BTW), 
> RSpec,
>>>>  scala, etc.?
>>>> 
>>>> 
>>>>>  3.)
>>>>>  'Integrated' tests vs 'Integration Tests'
>>>>>  We have 2 options to test our projects
>>>>> 
>>>>>  3.a.) create a full unit test in each module (e.g. 
> deltaspike/core/impl)
>>>>>  and add a profile for each and every server (maybe we can trim 
> this down
>>>>>  with pom imports?).
>>>>>  Then run the build with one after each other:
>>>>> 
>>>>>  $> mvn clean test -Powb
>>>>>  $> mvn clean test -Pweld
>>>>>  $> mvn clean test -Powb-tc
>>>>>  $> mvn clean test -Pweld-tc
>>>>>  $> mvn clean test -Pgf31
>>>>>  $> mvn clean test -Pas7
>>>>>  $> mvn clean test -Ptomee
>>>>> 
>>>>>  This might pretty much blow up our poms...
>>>>> 
>>>>> 
>>>>>  3.b) create a full unit test in each module (e.g. 
> deltaspike/core/impl)
>>>>>  and add only 2 profiles directly ('weld' and 
> 'owb' (probably resin later))
>>>>>  Then add a deltaspike/test/base/core with the very basic parent 
> stuff and
>>>>>  1 module for each integrated container, e.g.
>>>>>  deltaspike/test/weld-tc
>>>>>  deltaspike/test/owb-tc
>>>>>  deltaspike/test/weld-jetty
>>>>>  deltaspike/test/owb-jetty
>>>>>  deltaspike/test/tomee
>>>>>  deltaspike/test/gf31
>>>>>  deltaspike/test/geronimo
>>>>>  deltaspike/test/websphere
>>>>>  deltaspike/test/as7
>>>>>  deltaspike/test/as6
>>>>>  etc.
>>>>> 
>>>>>  The initial setup costs are higher, but it would be pretty easy 
> to add new
>>>>>  containers that way.
>>>>> 
>>>>>  Which route shall we take?
>>>>> 
>>>> 
>>>>  In Seam 3 we have an approach which is a little bit of a mixture of 
> both.
>>>>  We have regular unit tests (with mockito, stubs, etc) under
>>>>  impl/src/test/... and then we have a separate module called 
> testsuite which
>>>>  contained all the integration tests. We used pom imports to get all 
> the
>>>>  containers setup correctly and then had different profiles which 
> pulled in
>>>>  the poms and also configured surefire for that server. Inside the 
> code
>>>>  structure we separated the servers by package: common, jbossas, 
> glassfish,
>>>>  etc. That way all of the tests were in one location, they were easy 
> to
>>>>  execute, the tests stayed DRY, and the IDE wasn't much of a 
> problem. The
>>>>  two downsides we had were 1) the extra config in the pom and 2) you 
> can't
>>>>  run the full integration suite in one maven execution. We spent 
> quite a bit
>>>>  of time examining options and trying some of them out. This was the 
> best we
>>>>  came up with, and also the one Aslak thought would work best as 
> well.
>>>> 
>>>> 
>>>>>  LieGrue,
>>>>>  strub
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>>  --
>>>>  Jason Porter
>>>>  http://lightguard-jp.blogspot.com
>>>>  http://twitter.com/lightguardjp
>>>> 
>>>>  Software Engineer
>>>>  Open Source Advocate
>>>>  Author of Seam Catch - Next Generation Java Exception Handling
>>>> 
>>>>  PGP key id: 926CCFF5
>>>>  PGP key available at: keyserver.net, pgp.mit.edu
>>> 
>>> 
>>> 
>>> --
>>> Christian Kaltepoth
>>> Blog: http://chkal.blogspot.com/
>>> Twitter: http://twitter.com/chkal
>>> 
>>> 
>>> 
> 
> 
> 
> -- 
> Christian Kaltepoth
> Blog: http://chkal.blogspot.com/
> Twitter: http://twitter.com/chkal
> 

Mime
View raw message