maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Gudian <andreas.gud...@gmail.com>
Subject Re: maven surefire - selecting providers
Date Thu, 25 Jul 2013 18:34:10 GMT
Well, I guess you found a bug. TestNG should be able to handle JUnit4
tests, and thus the surefire TestNG provider should be able to hand over
the tests in a way that TestNG understands what we want it to do.

Would you be so kind and file a bug at
https://jira.codehaus.org/browse/SUREFIRE? No need to provide a test case,
I already hacked one together myself. ;)

Thanks,
Andreas



2013/7/25 Andreas Dolk <andreas.dolk.mobil@googlemail.com>

> I'm still struggling - I now think that testng really is in mixed mode and
> tries but fails to execute the jUnit test - maybe, because they use a
> special runner.
>
> Is there any way to disable the mixed mode for a maven build (entirely)?
> How can I pass that property "testng.mixed=false" to testng without having
> to write a testng.xml file (unless it is real trivial...)
>
> Best regards,
> Andreas
>
>
> 2013/7/24 Francesco Mari <mari.francesco@gmail.com>
>
> > I still wasn't able to reproduce your issue.
> >
> > Looks like TestNG is running in "mixed mode" [1][2]. The last missing
> > information is the version of JUnit and TestNG you are using. Can you
> > provide this piece of configuration?
> >
> > [1]: http://testng.org/doc/migrating.html
> > [2]: http://testng.org/doc/documentation-main.html#junit
> >
> >
> > 2013/7/24 Andreas Dolk <andreas.dolk.mobil@googlemail.com>
> >
> > > Hi Francesco,
> > >
> > > I'm using 2.15
> > >
> > > And here's the result from a test run, that's what happens. The tests
> are
> > > *only* jUnit tests. I've only replaced path and package names. BTW, the
> > > jUnit times are net execution times (unfair!!), testNG reports the
> total
> > > times (fair) ;) The other annoying part is that TestNG picks up far
> more
> > > classes then jUnit...
> > >
> > > The tests are auto-compiled from jnario specs (jnario.org) which
> > shouldn't
> > > make a difference - at the end it's classes compiled from java source
> > > files.
> > >
> > > Regards,
> > > Andreas
> > >
> > >
> > >
> > > > mvn -Dtest=JnSpec* test
> > >
> > > ...
> > >
> > > [INFO] --- maven-surefire-plugin:2.15:test (default-test) @
> a42-order-be
> > > ---
> > > [INFO] Surefire report directory: /<path>/target/surefire-reports
> > > [INFO] Using configured provider
> > > org.apache.maven.surefire.junitcore.JUnitCoreProvider
> > > [INFO] Using configured provider
> > > org.apache.maven.surefire.testng.TestNGProvider
> > > [INFO] parallel='none', perCoreThreadCount=true, threadCount=2,
> > > useUnlimitedThreads=false
> > >
> > > -------------------------------------------------------
> > >  T E S T S
> > > -------------------------------------------------------
> > >
> > > -------------------------------------------------------
> > >  T E S T S
> > > -------------------------------------------------------
> > > Running <package>.JnSpecBPMNProcessSpec
> > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.199
> > sec -
> > > in <package>.JnSpecBPMNProcessSpec
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec
> > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec
> > > Running
> > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec
> > > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.471
> > sec -
> > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec
> > > Running
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec
> > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.975
> > sec -
> > > in
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.225
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec
> > >
> > > Results :
> > >
> > > Tests run: 14, Failures: 0, Errors: 0, Skipped: 0
> > >
> > >
> > > -------------------------------------------------------
> > >  T E S T S
> > > -------------------------------------------------------
> > >
> > > -------------------------------------------------------
> > >  T E S T S
> > > -------------------------------------------------------
> > > Running <package>.JnSpecBPMNProcessSpec$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.595
> > sec -
> > > in <package>.JnSpecBPMNProcessSpec$1
> > > Running <package>.JnSpecBPMNProcessSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.693
> > sec -
> > > in <package>.JnSpecBPMNProcessSpec
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.658
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$1
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.683
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2$1
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.621
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$2
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.534
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$3
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.572
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1$1
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.142
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4$1
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.678
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$4
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.989
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5$1
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.461
> > sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec$5
> > > Running <package>.JnSpecCreateTheResultMessageOfACancellationSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.8
> sec -
> > > in <package>.JnSpecCreateTheResultMessageOfACancellationSpec
> > > Running
> > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.138
> > sec -
> > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$1
> > > Running
> > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.404
> > sec -
> > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$2
> > > Running
> > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.746
> > sec -
> > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$3
> > > Running
> > > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.242
> > sec -
> > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec$4
> > > Running
> > <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.571
> > sec -
> > > in <package>.JnSpecServicesToSupportCancellationOfItemsOfOneOrderSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.28
> sec
> > -
> > > in
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$1
> > > Running
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.334
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$2
> > > Running
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.363
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$3
> > > Running
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.148
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$4
> > > Running
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.506
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec$5
> > > Running
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.767
> > sec -
> > > in
> > >
> >
> <package>.JnSpecServicesToSupportCancellationOfItemsOfOnePurchaseOrderSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.883
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnAlreadyDeliveredOrderitemSpec
> > > Running
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.606
> > sec -
> > > in
> > >
> > >
> >
> <package>.JnSpecVerifyingTheCancellationMessageACancelRequestWithAnUndeliveredOrderitemSpec
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.275
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$1
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$2
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.147
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$2
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$3
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.246
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$3
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.514
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1$1
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.406
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1$1
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.614
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1$1
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.569
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4$1
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec$4
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.538
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec$4
> > > Running <package>.JnSpecVerifyingTheCancellationMessageSpec
> > > Configuring TestNG with: TestNG652Configurator
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.459
> > sec -
> > > in <package>.JnSpecVerifyingTheCancellationMessageSpec
> > >
> > > Results :
> > >
> > > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> > >
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > > [INFO] BUILD SUCCESS
> > > [INFO]
> > >
> ------------------------------------------------------------------------
> > >
> > >
> > >
> > > 2013/7/24 Francesco Mari <mari.francesco@gmail.com>
> > >
> > > > I tried to reproduce the problem with the version 2.14.1 of the
> plugin,
> > > but
> > > > everything works correctly. Which version of the plugin are you
> using?
> > > >
> > > >
> > > > 2013/7/24 Andreas Dolk <andreas.dolk.mobil@googlemail.com>
> > > >
> > > > > Sure, here we go:
> > > > >
> > > > >                 <plugin>
> > > > >                     <groupId>org.apache.maven.plugins</groupId>
> > > > >                     <artifactId>maven-surefire-plugin</artifactId>
> > > > >
> > <version>${version.maven-surefire-plugin}</version>
> > > > >                     <dependencies>
> > > > >                         <dependency>
> > > > >
> > > <groupId>org.apache.maven.surefire</groupId>
> > > > >
> <artifactId>surefire-junit47</artifactId>
> > > > >
> > > > > <version>${version.maven-surefire-plugin}</version>
> > > > >                         </dependency>
> > > > >                         <dependency>
> > > > >
> > > <groupId>org.apache.maven.surefire</groupId>
> > > > >
> <artifactId>surefire-testng</artifactId>
> > > > >
> > > > > <version>${version.maven-surefire-plugin}</version>
> > > > >                         </dependency>
> > > > >                     </dependencies>
> > > > >                     <configuration>
> > > > >                         <includes>
> > > > >                             <include>**/*Spec*.java</include>
> > > > >                             <include>**/*Feature.java</include>
> > > > >                             <include>**/*Test.java</include>
> > > > >                             <include>**/*TestCase.java</include>
> > > > >                         </includes>
> > > > >                         <forkCount>1</forkCount>
> > > > >                         <reuseForks>false</reuseForks>
> > > > >                     </configuration>
> > > > >                 </plugin>
> > > > >
> > > > >
> > > > >
> > > > > Andreas
> > > > >
> > > > >
> > > > > 2013/7/24 Francesco Mari <mari.francesco@gmail.com>
> > > > >
> > > > > > Can you provide your configuration for the m-surefire-p?
> > > > > >
> > > > > >
> > > > > > 2013/7/24 Andreas Dolk <andreas.dolk.mobil@googlemail.com>
> > > > > >
> > > > > > > Hello all,
> > > > > > >
> > > > > > > we have maven builds for projects that use both jUnit and
> testNG
> > > > tests.
> > > > > > So
> > > > > > > I configured the maven surefire plugin to use multiple
> providers
> > (
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html
> > > > > > > )
> > > > > > > and it works pretty well - apart from the problem, that
now
> > TestNG
> > > > > tries
> > > > > > to
> > > > > > > run every single jUnit test in the project. It takes about
2
> > > seconds
> > > > > for
> > > > > > > each test until it finds out that it has nothing to to
(only
> > jUnit
> > > > > > > annotations on the test class)
> > > > > > >
> > > > > > > The annoying part is that now some tests that failed with
jUnit
> > are
> > > > > > marked
> > > > > > > ok after TestNG didn't find a problem.
> > > > > > >
> > > > > > > Is there anyway to configure surefire or the providers
so that
> > they
> > > > > only
> > > > > > > run "their own" tests and ignore the others? A solution
where
> > >  rename
> > > > > the
> > > > > > > test to filter by names would be fine for me.
> > > > > > >
> > > > > > > Best regards,
> > > > > > >
> > > > > > >
> > > > > > > Andreas
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Andreas Dolk
> > > > >
> > > > > Zurmainerstraße 33
> > > > > D-54292 Trier
> > > > > Phone「+49 651 4362884」
> > > > > Mobile「+49 177 4970815」
> > > > > EMail「andreas.dolk.mobil@googlemail.com」
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Andreas Dolk
> > >
> > > Zurmainerstraße 33
> > > D-54292 Trier
> > > Phone「+49 651 4362884」
> > > Mobile「+49 177 4970815」
> > > EMail「andreas.dolk.mobil@googlemail.com」
> > >
> >
>
>
>
> --
> Andreas Dolk
>
> Zurmainerstraße 33
> D-54292 Trier
> Phone「+49 651 4362884」
> Mobile「+49 177 4970815」
> EMail「andreas.dolk.mobil@googlemail.com」
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message