deltaspike-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Struberg <strub...@yahoo.de>
Subject Re: basic decisions: test setup
Date Sat, 17 Dec 2011 16:19:59 GMT
true, we would need something like

@RunInEnvironments({Environment.EeLight, Environment.EeFull, Environment.Servlet})

@RunInEnvironments(Environment.JavaSE)
@RunInEnvironments(Environments.ALL)

LieGrue,
strub



----- Original Message -----
> From: Christian Kaltepoth <christian@kaltepoth.de>
> To: deltaspike-dev@incubator.apache.org
> Cc: 
> Sent: Saturday, December 17, 2011 5:04 PM
> Subject: Re: basic decisions: test setup
> 
> @Mark:
> 
> But what about tests that should run in EE-light, EE-full and Servlet
> but not in SE. Or tests that should run in EE-light/full but not in
> Servlet and SE. I think this could get very complex.
> 
> Christian
> 
> 
> 2011/12/17 Mark Struberg <struberg@yahoo.de>:
>>  We might solve this simply with a package naming rule. And then we 
> configure surefire to just run the tests with the respective package?
>> 
>>  Not the most advanced solution, but will work.
>> 
>> 
>>  LieGrue,
>>  strub
>> 
>> 
>> 
>>  ----- Original Message -----
>>>  From: Christian Kaltepoth <christian@kaltepoth.de>
>>>  To: deltaspike-dev@incubator.apache.org
>>>  Cc:
>>>  Sent: Saturday, December 17, 2011 4:43 PM
>>>  Subject: Re: basic decisions: test setup
>>> 
>>>  @Mark:
>>> 
>>>  Actually @TargetsContainer is an existing feature of Arquillian. It
>>>  allows to build container specific deployments.
>>> 
>>>  https://docs.jboss.org/author/display/ARQ/Multiple+Containers
>>> 
>>>  Unfortunately it won't fit our needs because the test completely 
> fails
>>>  if there is no matching deployment for the container the test is
>>>  executed against. So I guess we have to build something new for this.
>>> 
>>>  I really like your suggestion regarding the four environments. It
>>>  absolutely make sense to declare environment constraints (like SE,
>>>  Servlet, EE, etc.) instead of listing concrete containers.
>>> 
>>>  Christian
>>> 
>>> 
>>> 
>>>  2011/12/17 Mark Struberg <struberg@yahoo.de>:
>>>>   Hi Christian!
>>>> 
>>>> 
>>>>   @TargetContainer sounds good, but I'd rather make it a
>>>> 
>>>> 
>>>>   @TargetEnvironment()
>>>> 
>>>>   with values: SE, Servlet, EE-light, EE-full
>>>> 
>>>>   The excludes should be done in the single test integration 
> modules, and the
>>>  goal should be to get down to zero excluded tests for all containers.
>>>> 
>>>> 
>>>>   @Jason: that mixture for the tests looks fine to me as well. Can 
> you please
>>>  hack a sample structure in the DeltaSpikePlayground git repo?
>>>> 
>>>> 
>>>>   LieGrue,
>>>>   strub
>>>> 
>>>> 
>>>> 
>>>> 
>>>>>  ________________________________
>>>>>   From: Christian Kaltepoth <christian@kaltepoth.de>
>>>>>  To: deltaspike-dev@incubator.apache.org
>>>>>  Sent: Saturday, December 17, 2011 7:55 AM
>>>>>  Subject: Re: basic decisions: test setup
>>>>> 
>>>>>  I would like to add some "non-binding" comments here 
> regarding
>>>  3).
>>>>> 
>>>>>  Actually I really like the concept Jason described and which is
>>>>>  currently used in Seam3. Having worked with it a bit I can 
> confirm
>>>>>  that it runs fine. And somehow it's the best of both worlds 
> (in
>>>  regard
>>>>>  to 3a vs 3b).
>>>>> 
>>>>>  That the full integration test suite with all containers cannot 
> run in
>>>>>  a single Maven invocation is IMHO not a real problem. We could 
> write a
>>>>>  simple script for this.
>>>>> 
>>>>>  The only thing that could be optimized in this setup is the 
> separation
>>>>>  of container dependent tests via package name. But perhaps this 
> could
>>>>>  be solved by Arquillian? What I would really like to see is the
>>>>>  ability to configure a test so it is only executed in a 
> specific
>>>>>  container (something similar to @TargetsContainer). This way 
> tests
>>>>>  that require to be run in a full container could be executed 
> only in
>>>>>  JBoss and Glassfish but would be skipped in embedded Weld.
>>>>> 
>>>>>  Christian
>>>>> 
>>>>> 
>>>>>  2011/12/16 Jason Porter <lightguard.jp@gmail.com>:
>>>>>>   Comments inline.
>>>>>> 
>>>>>>   On Fri, Dec 16, 2011 at 01:38, Mark Struberg
>>>  <struberg@yahoo.de> wrote:
>>>>>> 
>>>>>>>   Hi!
>>>>>>> 
>>>>>>>   Another round of discussion - this time about how to 
> setup our
>>>  unit
>>>>>>>   testing and integration testing.
>>>>>>> 
>>>>>>>   I think it's pretty much clear that we will gonna 
> use
>>>  Arquillian for all
>>>>>>>   our tests. That was also fundamental part of the 
> incubator
>>>  proposal.
>>>>>>>   And instead of inventing yet another abstraction 
> layer, I'd
>>>  favour to just
>>>>>>>   use Arquillian and contribute all things we need to 
> this
>>>  project.
>>>>>>>   So here a formal vote
>>>>>>> 
>>>>>>>   1.)
>>>>>>> 
>>>>>>>   for using Arquillian as test integration framework.
>>>>>>>    +1 from me
>>>>>>> 
>>>>>> 
>>>>>>   +1 from me
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>>   2.)
>>>>>>> 
>>>>>>>   What do we like to use from the unit tests itself? 
> TestNG or
>>>  JUnit?
>>>>>>>   JUnit has better Arquillian integration and TestNG is 
> better
>>>  for 'real
>>>>>>>   world' projects, because it allows to define test
>>>  dependencies.
>>>>>>>   So which one to take?
>>>>>>>   But we should definitely only use 1 of the two 
> exclusively!
>>>>>>> 
>>>>>> 
>>>>>>   +1 for using one exclusively
>>>>>> 
>>>>>>   As for which, I'm also +0 I like both for different 
> reasons,
>>>  but JUnit is
>>>>>>   certainly the easier of the two to use with Arquillian.
>>>>>> 
>>>>>>   Also, are we open to having other languages for tests or 
> using
>>>  other
>>>>>>   testing structures such as spock (it works with 
> Arquillian, BTW),
>>>  RSpec,
>>>>>>   scala, etc.?
>>>>>> 
>>>>>> 
>>>>>>>   3.)
>>>>>>>   'Integrated' tests vs 'Integration 
> Tests'
>>>>>>>   We have 2 options to test our projects
>>>>>>> 
>>>>>>>   3.a.) create a full unit test in each module (e.g.
>>>  deltaspike/core/impl)
>>>>>>>   and add a profile for each and every server (maybe we 
> can trim
>>>  this down
>>>>>>>   with pom imports?).
>>>>>>>   Then run the build with one after each other:
>>>>>>> 
>>>>>>>   $> mvn clean test -Powb
>>>>>>>   $> mvn clean test -Pweld
>>>>>>>   $> mvn clean test -Powb-tc
>>>>>>>   $> mvn clean test -Pweld-tc
>>>>>>>   $> mvn clean test -Pgf31
>>>>>>>   $> mvn clean test -Pas7
>>>>>>>   $> mvn clean test -Ptomee
>>>>>>> 
>>>>>>>   This might pretty much blow up our poms...
>>>>>>> 
>>>>>>> 
>>>>>>>   3.b) create a full unit test in each module (e.g.
>>>  deltaspike/core/impl)
>>>>>>>   and add only 2 profiles directly ('weld' and
>>>  'owb' (probably resin later))
>>>>>>>   Then add a deltaspike/test/base/core with the very 
> basic parent
>>>  stuff and
>>>>>>>   1 module for each integrated container, e.g.
>>>>>>>   deltaspike/test/weld-tc
>>>>>>>   deltaspike/test/owb-tc
>>>>>>>   deltaspike/test/weld-jetty
>>>>>>>   deltaspike/test/owb-jetty
>>>>>>>   deltaspike/test/tomee
>>>>>>>   deltaspike/test/gf31
>>>>>>>   deltaspike/test/geronimo
>>>>>>>   deltaspike/test/websphere
>>>>>>>   deltaspike/test/as7
>>>>>>>   deltaspike/test/as6
>>>>>>>   etc.
>>>>>>> 
>>>>>>>   The initial setup costs are higher, but it would be 
> pretty easy
>>>  to add new
>>>>>>>   containers that way.
>>>>>>> 
>>>>>>>   Which route shall we take?
>>>>>>> 
>>>>>> 
>>>>>>   In Seam 3 we have an approach which is a little bit of a 
> mixture of
>>>  both.
>>>>>>   We have regular unit tests (with mockito, stubs, etc) 
> under
>>>>>>   impl/src/test/... and then we have a separate module 
> called
>>>  testsuite which
>>>>>>   contained all the integration tests. We used pom imports 
> to get all
>>>  the
>>>>>>   containers setup correctly and then had different profiles 
> which
>>>  pulled in
>>>>>>   the poms and also configured surefire for that server. 
> Inside the
>>>  code
>>>>>>   structure we separated the servers by package: common, 
> jbossas,
>>>  glassfish,
>>>>>>   etc. That way all of the tests were in one location, they 
> were easy
>>>  to
>>>>>>   execute, the tests stayed DRY, and the IDE wasn't much 
> of a
>>>  problem. The
>>>>>>   two downsides we had were 1) the extra config in the pom 
> and 2) you
>>>  can't
>>>>>>   run the full integration suite in one maven execution. We 
> spent
>>>  quite a bit
>>>>>>   of time examining options and trying some of them out. 
> This was the
>>>  best we
>>>>>>   came up with, and also the one Aslak thought would work 
> best as
>>>  well.
>>>>>> 
>>>>>> 
>>>>>>>   LieGrue,
>>>>>>>   strub
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>   --
>>>>>>   Jason Porter
>>>>>>   http://lightguard-jp.blogspot.com
>>>>>>   http://twitter.com/lightguardjp
>>>>>> 
>>>>>>   Software Engineer
>>>>>>   Open Source Advocate
>>>>>>   Author of Seam Catch - Next Generation Java Exception 
> Handling
>>>>>> 
>>>>>>   PGP key id: 926CCFF5
>>>>>>   PGP key available at: keyserver.net, pgp.mit.edu
>>>>> 
>>>>> 
>>>>> 
>>>>>  --
>>>>>  Christian Kaltepoth
>>>>>  Blog: http://chkal.blogspot.com/
>>>>>  Twitter: http://twitter.com/chkal
>>>>> 
>>>>> 
>>>>> 
>>> 
>>> 
>>> 
>>>  --
>>>  Christian Kaltepoth
>>>  Blog: http://chkal.blogspot.com/
>>>  Twitter: http://twitter.com/chkal
>>> 
> 
> 
> 
> -- 
> Christian Kaltepoth
> Blog: http://chkal.blogspot.com/
> Twitter: http://twitter.com/chkal
> 

Mime
View raw message