deltaspike-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Kaltepoth <christ...@kaltepoth.de>
Subject Re: basic decisions: test setup
Date Sat, 17 Dec 2011 16:04:25 GMT
@Mark:

But what about tests that should run in EE-light, EE-full and Servlet
but not in SE. Or tests that should run in EE-light/full but not in
Servlet and SE. I think this could get very complex.

Christian


2011/12/17 Mark Struberg <struberg@yahoo.de>:
> We might solve this simply with a package naming rule. And then we configure surefire
to just run the tests with the respective package?
>
> Not the most advanced solution, but will work.
>
>
> LieGrue,
> strub
>
>
>
> ----- Original Message -----
>> From: Christian Kaltepoth <christian@kaltepoth.de>
>> To: deltaspike-dev@incubator.apache.org
>> Cc:
>> Sent: Saturday, December 17, 2011 4:43 PM
>> Subject: Re: basic decisions: test setup
>>
>> @Mark:
>>
>> Actually @TargetsContainer is an existing feature of Arquillian. It
>> allows to build container specific deployments.
>>
>> https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>>
>> Unfortunately it won't fit our needs because the test completely fails
>> if there is no matching deployment for the container the test is
>> executed against. So I guess we have to build something new for this.
>>
>> I really like your suggestion regarding the four environments. It
>> absolutely make sense to declare environment constraints (like SE,
>> Servlet, EE, etc.) instead of listing concrete containers.
>>
>> Christian
>>
>>
>>
>> 2011/12/17 Mark Struberg <struberg@yahoo.de>:
>>>  Hi Christian!
>>>
>>>
>>>  @TargetContainer sounds good, but I'd rather make it a
>>>
>>>
>>>  @TargetEnvironment()
>>>
>>>  with values: SE, Servlet, EE-light, EE-full
>>>
>>>  The excludes should be done in the single test integration modules, and the
>> goal should be to get down to zero excluded tests for all containers.
>>>
>>>
>>>  @Jason: that mixture for the tests looks fine to me as well. Can you please
>> hack a sample structure in the DeltaSpikePlayground git repo?
>>>
>>>
>>>  LieGrue,
>>>  strub
>>>
>>>
>>>
>>>
>>>> ________________________________
>>>>  From: Christian Kaltepoth <christian@kaltepoth.de>
>>>> To: deltaspike-dev@incubator.apache.org
>>>> Sent: Saturday, December 17, 2011 7:55 AM
>>>> Subject: Re: basic decisions: test setup
>>>>
>>>> I would like to add some "non-binding" comments here regarding
>> 3).
>>>>
>>>> Actually I really like the concept Jason described and which is
>>>> currently used in Seam3. Having worked with it a bit I can confirm
>>>> that it runs fine. And somehow it's the best of both worlds (in
>> regard
>>>> to 3a vs 3b).
>>>>
>>>> That the full integration test suite with all containers cannot run in
>>>> a single Maven invocation is IMHO not a real problem. We could write a
>>>> simple script for this.
>>>>
>>>> The only thing that could be optimized in this setup is the separation
>>>> of container dependent tests via package name. But perhaps this could
>>>> be solved by Arquillian? What I would really like to see is the
>>>> ability to configure a test so it is only executed in a specific
>>>> container (something similar to @TargetsContainer). This way tests
>>>> that require to be run in a full container could be executed only in
>>>> JBoss and Glassfish but would be skipped in embedded Weld.
>>>>
>>>> Christian
>>>>
>>>>
>>>> 2011/12/16 Jason Porter <lightguard.jp@gmail.com>:
>>>>>  Comments inline.
>>>>>
>>>>>  On Fri, Dec 16, 2011 at 01:38, Mark Struberg
>> <struberg@yahoo.de> wrote:
>>>>>
>>>>>>  Hi!
>>>>>>
>>>>>>  Another round of discussion - this time about how to setup our
>> unit
>>>>>>  testing and integration testing.
>>>>>>
>>>>>>  I think it's pretty much clear that we will gonna use
>> Arquillian for all
>>>>>>  our tests. That was also fundamental part of the incubator
>> proposal.
>>>>>>  And instead of inventing yet another abstraction layer, I'd
>> favour to just
>>>>>>  use Arquillian and contribute all things we need to this
>> project.
>>>>>>  So here a formal vote
>>>>>>
>>>>>>  1.)
>>>>>>
>>>>>>  for using Arquillian as test integration framework.
>>>>>>   +1 from me
>>>>>>
>>>>>
>>>>>  +1 from me
>>>>>
>>>>>
>>>>>>
>>>>>>  2.)
>>>>>>
>>>>>>  What do we like to use from the unit tests itself? TestNG or
>> JUnit?
>>>>>>  JUnit has better Arquillian integration and TestNG is better
>> for 'real
>>>>>>  world' projects, because it allows to define test
>> dependencies.
>>>>>>  So which one to take?
>>>>>>  But we should definitely only use 1 of the two exclusively!
>>>>>>
>>>>>
>>>>>  +1 for using one exclusively
>>>>>
>>>>>  As for which, I'm also +0 I like both for different reasons,
>> but JUnit is
>>>>>  certainly the easier of the two to use with Arquillian.
>>>>>
>>>>>  Also, are we open to having other languages for tests or using
>> other
>>>>>  testing structures such as spock (it works with Arquillian, BTW),
>> RSpec,
>>>>>  scala, etc.?
>>>>>
>>>>>
>>>>>>  3.)
>>>>>>  'Integrated' tests vs 'Integration Tests'
>>>>>>  We have 2 options to test our projects
>>>>>>
>>>>>>  3.a.) create a full unit test in each module (e.g.
>> deltaspike/core/impl)
>>>>>>  and add a profile for each and every server (maybe we can trim
>> this down
>>>>>>  with pom imports?).
>>>>>>  Then run the build with one after each other:
>>>>>>
>>>>>>  $> mvn clean test -Powb
>>>>>>  $> mvn clean test -Pweld
>>>>>>  $> mvn clean test -Powb-tc
>>>>>>  $> mvn clean test -Pweld-tc
>>>>>>  $> mvn clean test -Pgf31
>>>>>>  $> mvn clean test -Pas7
>>>>>>  $> mvn clean test -Ptomee
>>>>>>
>>>>>>  This might pretty much blow up our poms...
>>>>>>
>>>>>>
>>>>>>  3.b) create a full unit test in each module (e.g.
>> deltaspike/core/impl)
>>>>>>  and add only 2 profiles directly ('weld' and
>> 'owb' (probably resin later))
>>>>>>  Then add a deltaspike/test/base/core with the very basic parent
>> stuff and
>>>>>>  1 module for each integrated container, e.g.
>>>>>>  deltaspike/test/weld-tc
>>>>>>  deltaspike/test/owb-tc
>>>>>>  deltaspike/test/weld-jetty
>>>>>>  deltaspike/test/owb-jetty
>>>>>>  deltaspike/test/tomee
>>>>>>  deltaspike/test/gf31
>>>>>>  deltaspike/test/geronimo
>>>>>>  deltaspike/test/websphere
>>>>>>  deltaspike/test/as7
>>>>>>  deltaspike/test/as6
>>>>>>  etc.
>>>>>>
>>>>>>  The initial setup costs are higher, but it would be pretty easy
>> to add new
>>>>>>  containers that way.
>>>>>>
>>>>>>  Which route shall we take?
>>>>>>
>>>>>
>>>>>  In Seam 3 we have an approach which is a little bit of a mixture of
>> both.
>>>>>  We have regular unit tests (with mockito, stubs, etc) under
>>>>>  impl/src/test/... and then we have a separate module called
>> testsuite which
>>>>>  contained all the integration tests. We used pom imports to get all
>> the
>>>>>  containers setup correctly and then had different profiles which
>> pulled in
>>>>>  the poms and also configured surefire for that server. Inside the
>> code
>>>>>  structure we separated the servers by package: common, jbossas,
>> glassfish,
>>>>>  etc. That way all of the tests were in one location, they were easy
>> to
>>>>>  execute, the tests stayed DRY, and the IDE wasn't much of a
>> problem. The
>>>>>  two downsides we had were 1) the extra config in the pom and 2) you
>> can't
>>>>>  run the full integration suite in one maven execution. We spent
>> quite a bit
>>>>>  of time examining options and trying some of them out. This was the
>> best we
>>>>>  came up with, and also the one Aslak thought would work best as
>> well.
>>>>>
>>>>>
>>>>>>  LieGrue,
>>>>>>  strub
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>  --
>>>>>  Jason Porter
>>>>>  http://lightguard-jp.blogspot.com
>>>>>  http://twitter.com/lightguardjp
>>>>>
>>>>>  Software Engineer
>>>>>  Open Source Advocate
>>>>>  Author of Seam Catch - Next Generation Java Exception Handling
>>>>>
>>>>>  PGP key id: 926CCFF5
>>>>>  PGP key available at: keyserver.net, pgp.mit.edu
>>>>
>>>>
>>>>
>>>> --
>>>> Christian Kaltepoth
>>>> Blog: http://chkal.blogspot.com/
>>>> Twitter: http://twitter.com/chkal
>>>>
>>>>
>>>>
>>
>>
>>
>> --
>> Christian Kaltepoth
>> Blog: http://chkal.blogspot.com/
>> Twitter: http://twitter.com/chkal
>>



-- 
Christian Kaltepoth
Blog: http://chkal.blogspot.com/
Twitter: http://twitter.com/chkal

Mime
View raw message