Hi All,

The only down side I can see to testng is the reliance on surefire 2.8-SNAPSHOT.

The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3-SNAPSHOT from http://people.apache.org/maven-snapshot-repository and everything seemed to work fine. Jason could you try that out so we can get rid of the 2.8-SNAPSHOT dependency?

Apart from that I think testng is the way to go.

TTFN,

-bd-

------------------ begin patch ---------------------------
Index: pom.xml
===================================================================
--- pom.xml     (revision 439918)
+++ pom.xml     (working copy)
@@ -93,7 +93,7 @@
                 <plugin>
                     <groupId>org.apache.maven.plugins</groupId>
                     <artifactId>maven-surefire-plugin</artifactId>
-                    <version>2.8-SNAPSHOT</version>
+                    <version>2.3-SNAPSHOT</version>
                 </plugin>
             </plugins>
         </pluginManagement>
@@ -116,14 +116,4 @@
         </repository>
     </repositories>
    
-    <!--
-    NOTE: This is the repository where maven-surefire-plugin:2.8-SNAPSHOT comes from.
-    -->
-    <pluginRepositories>
-        <pluginRepository>
-            <id>tapestry.javaforge</id>
-            <url>http://howardlewisship.com/repository</url>
-        </pluginRepository>
-    </pluginRepositories>
-
</project>
---------------------- end patch--------------------------------------



On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:

Ya, I see this from time to time... not really sure why it pops up sometimes and not at others.  Its easy enough to fix... but I think that its much easier with ng.

Still talking to maven folks about getting a release of surefire that works with testng + jdk14, but the 2.8-SNAPSHOT works for now...

--jason


On 9/3/06, Bill Dudney <bdudney@apache.org> wrote:
So you got around this? This was not happening for me so I'm not sure
what is going on, it appears from the next message that you got it to
work.

I did see an UndeclaredThrowableException a time or two but it was
always because the selenium server was not running.

I'm going to dig into your testng stuff in the am and will post
thoughts.

TTFN,

bd-
On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:

> FYI, I applied your patch... and surefire barfs as I thought it might:
>
> <snip>
> java.lang.reflect.UndeclaredThrowableException
>         at $Proxy0.addError(Unknown Source)
>         at junit.framework.TestResult.addError(TestResult.java:36)
>         at junit.framework.TestResult.runProtected(TestResult.java:
> 133)
>         at junit.extensions.TestSetup.run(TestSetup.java:23)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java :39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:324)
>         at org.apache.maven.surefire.junit.JUnitTestSet.execute
> (JUnitTestSet.java:210)
>         at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTest
> Set(AbstractDirectoryTestSuite.java:135)
>         at
> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
> (AbstractDirectoryTestSuite.java:122)
>         at org.apache.maven.surefire.Surefire.run(Surefire.java:129)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke (Method.java:324)
>         at
> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
> (SurefireBooter.java:262)
>         at org.apache.maven.surefire.booter.SurefireBooter.main
> (SurefireBooter.java:787)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getName()
>         at java.lang.Class.getMethod(Class.java:986)
>         at
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getStack
> TraceWriter(TestListenerInvocationHandler.java:171)
>         at
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handleAd
> dError(TestListenerInvocationHandler.java:160)
>         at
> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invoke
> (TestListenerInvocationHandler.java:134)
>         ... 18 more
> </snip>
>
> --jason
>
>
> On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>
>> Hi Jason,
>>
>> This is very cool indeed, thanks for putting the prototype
>> together. I just submitted a patch that makes firefox pop up only
>> once to run all the tests (it actually pops up twice, once during
>> selenium.start() and then once for the test suite but the first
>> time is for a split second).
>>
>> http://issues.apache.org/jira/browse/GERONIMO-2374
>>
>> It uses a JUnit TestDecorator to run setup only once for the whole
>> suite.
>>
>> IMO until we move to JDK 5 its not a great idea to move to TestNG,
>> you will get some benefits for sure but the source has to be
>> parsed at runtime to get at the meta-data ( i.e. the info about the
>> tests). Krufty IMO.
>>
>> The patch has a comment about invalid XHTML.  I believe the
>> invalid XHTML part of the console is preventing the XPath find
>> from working.
>>
>> TTFN,
>>
>> -bd-
>>
>>
>> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>>
>>> I've checked in a new top-level testsuite module and a few new
>>> plugins to support it.  This is only the start and I expect it to
>>> change over the next few weeks (er maybe months) as momentum
>>> starts to pick up.
>>>
>>> I looked into Cargo, and while I think we should eventually use
>>> it to start/stop the server... asis now its broke.  We need to
>>> provide a very simple and consistent way for Cargo to invoke some
>>> operations via JMX, which I hinted to in previous emails.  Once
>>> that is done, we can patch Cargo and hopefully get that committed
>>> to support G... but in the mean time I whipped up a simple start
>>> & stop mojo that uses Ant to start and stop the server.  Its
>>> very, very crude and we need to fix stuff like logging output etc.
>>>
>>> I also played around a little with Selenium to make some tests
>>> for the console.  It wants a special server process started, so I
>>> created a selenium plugin which currently only starts that server
>>> in... well an icky way, but should work for the moment.  Need to
>>> ask the m2 folks how to do this better.
>>>
>>> I created a console-testsuite module, which sets up the G server
>>> and Selenium server in pre-integration-test, and then uses the
>>> maven-invoker-plugin to invoke the basic module to actually run
>>> the tests.  Only one test class right now, SimpleLoginTest, which
>>> does just that... logs in, logs in and then out and then another
>>> that logs in and clicks some links.
>>>
>>> I've got it all running from one mvn command and so far, on my
>>> mac, it works fine.  I'd imagine due to the location of the
>>> assembly that Windows will not be able to run this... we'll need
>>> to fix this later.  But those of you lucky enough to have a Linux
>>> box or a mac, you should be able to:
>>>
>>>     mvn clean && mvn install
>>>     cd testsuite
>>>     mvn
>>>
>>> (the testsuite is not hooked up to the default build)
>>>
>>> And you should see Firefox popup a few times as Selenium goes
>>> about clicking on stuff.  Right now its only really doing
>>> validation of the console page's titles... but if everything
>>> works, then the invoker should not complain and the build should
>>> complete.
>>>
>>> A bunch more needs to be done... like creating a common test
>>> support class for selenium tests... and probably switching these
>>> tests to use testng, so that the selenium setup is done once, not
>>> for each test... as it does now... so expect Firefox to start and
>>> stop a few times.  We need to have some helpers to pick out the
>>> console contents easily so they can be validated... I read that
>>> selenium supports xpath or something, but I have not actually
>>> tried any of that.
>>>
>>> This is just a very, very crude proof of concept... needs to have
>>> reporting features added, capture logs, blah, blah, blah.
>>>
>>> Anyways, I committed this so we have a common base to work
>>> patches off of.
>>>
>>> One thing to note about the Selenium IDE, is that sometimes it
>>> does not put into a waitForPageToLoad(), like if you record a
>>> login to the console... you need to manually add that wait after
>>> the button is clicked or it will hose the test.
>>>
>>> I'm not married to any of this either... I just wanted something
>>> that worked... or kinda worked to show that its not going to be
>>> that hard to make some rich integration tests.
>>>
>>> So, comments, suggestions, patches... all welcome.
>>>
>>> --jason
>>>
>>>
>>
>