geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Dillon <ja...@planet57.com>
Subject Re: Testsuite module, with basic/crude Selenium support
Date Mon, 04 Sep 2006 17:55:51 GMT
K, no worries.  Though... looks like some time is going to be put  
into fixing up surefire in the week or so, so I don't think we will  
need 2.8-SNAPSHOT for too long.

--jason


On Sep 4, 2006, at 10:53 AM, Bill Dudney wrote:

> Hey Jason,
>
> Sorry for the false alarm, the tests don't run. Looks like 2.8- 
> SNAPSHOT is what we will have to use until the surefire folks can  
> do an update.
>
> TTFN,
>
> -bd-
>
> On Sep 4, 2006, at 10:34 AM, Jason Dillon wrote:
>
>> Did the tests actually run?  Make sure that it actually ran tests  
>> for basic/*
>>
>> --jason
>>
>>
>> On Sep 4, 2006, at 9:33 AM, Bill Dudney wrote:
>>
>>> Hi All,
>>>
>>> The only down side I can see to testng is the reliance on  
>>> surefire 2.8-SNAPSHOT.
>>>
>>> The whole 2.8-SNAPSHOT thing seems weird so I tried out 2.3- 
>>> SNAPSHOT from http://people.apache.org/maven-snapshot-repository  
>>> and everything seemed to work fine. Jason could you try that out  
>>> so we can get rid of the 2.8-SNAPSHOT dependency?
>>>
>>> Apart from that I think testng is the way to go.
>>>
>>> TTFN,
>>>
>>> -bd-
>>>
>>> ------------------ begin patch ---------------------------
>>> Index: pom.xml
>>> ===================================================================
>>> --- pom.xml     (revision 439918)
>>> +++ pom.xml     (working copy)
>>> @@ -93,7 +93,7 @@
>>>                  <plugin>
>>>                      <groupId>org.apache.maven.plugins</groupId>
>>>                      <artifactId>maven-surefire-plugin</artifactId>
>>> -                    <version>2.8-SNAPSHOT</version>
>>> +                    <version>2.3-SNAPSHOT</version>
>>>                  </plugin>
>>>              </plugins>
>>>          </pluginManagement>
>>> @@ -116,14 +116,4 @@
>>>          </repository>
>>>      </repositories>
>>>
>>> -    <!--
>>> -    NOTE: This is the repository where maven-surefire-plugin:2.8- 
>>> SNAPSHOT comes from.
>>> -    -->
>>> -    <pluginRepositories>
>>> -        <pluginRepository>
>>> -            <id>tapestry.javaforge</id>
>>> -            <url>http://howardlewisship.com/repository</url>
>>> -        </pluginRepository>
>>> -    </pluginRepositories>
>>> -
>>> </project>
>>> ---------------------- end  
>>> patch--------------------------------------
>>>
>>>
>>>
>>> On Sep 4, 2006, at 12:01 AM, Jason Dillon wrote:
>>>
>>>> Ya, I see this from time to time... not really sure why it pops  
>>>> up sometimes and not at others.  Its easy enough to fix... but I  
>>>> think that its much easier with ng.
>>>>
>>>> Still talking to maven folks about getting a release of surefire  
>>>> that works with testng + jdk14, but the 2.8-SNAPSHOT works for  
>>>> now...
>>>>
>>>> --jason
>>>>
>>>>
>>>> On 9/3/06, Bill Dudney <bdudney@apache.org> wrote:
>>>> So you got around this? This was not happening for me so I'm not  
>>>> sure
>>>> what is going on, it appears from the next message that you got  
>>>> it to
>>>> work.
>>>>
>>>> I did see an UndeclaredThrowableException a time or two but it was
>>>> always because the selenium server was not running.
>>>>
>>>> I'm going to dig into your testng stuff in the am and will post
>>>> thoughts.
>>>>
>>>> TTFN,
>>>>
>>>> bd-
>>>> On Sep 3, 2006, at 6:03 AM, Jason Dillon wrote:
>>>>
>>>> > FYI, I applied your patch... and surefire barfs as I thought  
>>>> it might:
>>>> >
>>>> > <snip>
>>>> > java.lang.reflect.UndeclaredThrowableException
>>>> >         at $Proxy0.addError(Unknown Source)
>>>> >         at junit.framework.TestResult.addError(TestResult.java: 
>>>> 36)
>>>> >         at junit.framework.TestResult.runProtected 
>>>> (TestResult.java:
>>>> > 133)
>>>> >         at junit.extensions.TestSetup.run(TestSetup.java:23)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native  
>>>> Method)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke
>>>> > (NativeMethodAccessorImpl.java :39)
>>>> >         at sun.reflect.DelegatingMethodAccessorImpl.invoke
>>>> > (DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke(Method.java:324)
>>>> >         at org.apache.maven.surefire.junit.JUnitTestSet.execute
>>>> > (JUnitTestSet.java:210)
>>>> >         at
>>>> >  
>>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeT 
>>>> est
>>>> > Set(AbstractDirectoryTestSuite.java:135)
>>>> >         at
>>>> >  
>>>> org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
>>>> > (AbstractDirectoryTestSuite.java:122)
>>>> >         at org.apache.maven.surefire.Surefire.run 
>>>> (Surefire.java:129)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native  
>>>> Method)
>>>> >         at sun.reflect.NativeMethodAccessorImpl.invoke
>>>> > (NativeMethodAccessorImpl.java:39)
>>>> >         at sun.reflect.DelegatingMethodAccessorImpl.invoke
>>>> > (DelegatingMethodAccessorImpl.java:25)
>>>> >         at java.lang.reflect.Method.invoke (Method.java:324)
>>>> >         at
>>>> >  
>>>> org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess
>>>> > (SurefireBooter.java:262)
>>>> >         at org.apache.maven.surefire.booter.SurefireBooter.main
>>>> > (SurefireBooter.java:787)
>>>> > Caused by: java.lang.NoSuchMethodException:
>>>> >  
>>>> org.apache.geronimo.testsuite.console.StartSeleniumDecorator.getNam 
>>>> e()
>>>> >         at java.lang.Class.getMethod(Class.java:986)
>>>> >         at
>>>> >  
>>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.getSt 
>>>> ack
>>>> > TraceWriter(TestListenerInvocationHandler.java:171)
>>>> >         at
>>>> >  
>>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.handl 
>>>> eAd
>>>> > dError(TestListenerInvocationHandler.java:160)
>>>> >         at
>>>> >  
>>>> org.apache.maven.surefire.junit.TestListenerInvocationHandler.invok 
>>>> e
>>>> > (TestListenerInvocationHandler.java:134)
>>>> >         ... 18 more
>>>> > </snip>
>>>> >
>>>> > --jason
>>>> >
>>>> >
>>>> > On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:
>>>> >
>>>> >> Hi Jason,
>>>> >>
>>>> >> This is very cool indeed, thanks for putting the prototype
>>>> >> together. I just submitted a patch that makes firefox pop up  
>>>> only
>>>> >> once to run all the tests (it actually pops up twice, once  
>>>> during
>>>> >> selenium.start() and then once for the test suite but the first
>>>> >> time is for a split second).
>>>> >>
>>>> >> http://issues.apache.org/jira/browse/GERONIMO-2374
>>>> >>
>>>> >> It uses a JUnit TestDecorator to run setup only once for the  
>>>> whole
>>>> >> suite.
>>>> >>
>>>> >> IMO until we move to JDK 5 its not a great idea to move to  
>>>> TestNG,
>>>> >> you will get some benefits for sure but the source has to be
>>>> >> parsed at runtime to get at the meta-data ( i.e. the info  
>>>> about the
>>>> >> tests). Krufty IMO.
>>>> >>
>>>> >> The patch has a comment about invalid XHTML.  I believe the
>>>> >> invalid XHTML part of the console is preventing the XPath find
>>>> >> from working.
>>>> >>
>>>> >> TTFN,
>>>> >>
>>>> >> -bd-
>>>> >>
>>>> >>
>>>> >> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>>>> >>
>>>> >>> I've checked in a new top-level testsuite module and a few new
>>>> >>> plugins to support it.  This is only the start and I expect
 
>>>> it to
>>>> >>> change over the next few weeks (er maybe months) as momentum
>>>> >>> starts to pick up.
>>>> >>>
>>>> >>> I looked into Cargo, and while I think we should eventually
use
>>>> >>> it to start/stop the server... asis now its broke.  We need
to
>>>> >>> provide a very simple and consistent way for Cargo to invoke
 
>>>> some
>>>> >>> operations via JMX, which I hinted to in previous emails.  Once
>>>> >>> that is done, we can patch Cargo and hopefully get that  
>>>> committed
>>>> >>> to support G... but in the mean time I whipped up a simple 

>>>> start
>>>> >>> & stop mojo that uses Ant to start and stop the server.
 Its
>>>> >>> very, very crude and we need to fix stuff like logging  
>>>> output etc.
>>>> >>>
>>>> >>> I also played around a little with Selenium to make some tests
>>>> >>> for the console.  It wants a special server process started,
 
>>>> so I
>>>> >>> created a selenium plugin which currently only starts that 

>>>> server
>>>> >>> in... well an icky way, but should work for the moment.   
>>>> Need to
>>>> >>> ask the m2 folks how to do this better.
>>>> >>>
>>>> >>> I created a console-testsuite module, which sets up the G  
>>>> server
>>>> >>> and Selenium server in pre-integration-test, and then uses the
>>>> >>> maven-invoker-plugin to invoke the basic module to actually
run
>>>> >>> the tests.  Only one test class right now, SimpleLoginTest,
 
>>>> which
>>>> >>> does just that... logs in, logs in and then out and then  
>>>> another
>>>> >>> that logs in and clicks some links.
>>>> >>>
>>>> >>> I've got it all running from one mvn command and so far, on
my
>>>> >>> mac, it works fine.  I'd imagine due to the location of the
>>>> >>> assembly that Windows will not be able to run this... we'll
 
>>>> need
>>>> >>> to fix this later.  But those of you lucky enough to have a
 
>>>> Linux
>>>> >>> box or a mac, you should be able to:
>>>> >>>
>>>> >>>     mvn clean && mvn install
>>>> >>>     cd testsuite
>>>> >>>     mvn
>>>> >>>
>>>> >>> (the testsuite is not hooked up to the default build)
>>>> >>>
>>>> >>> And you should see Firefox popup a few times as Selenium goes
>>>> >>> about clicking on stuff.  Right now its only really doing
>>>> >>> validation of the console page's titles... but if everything
>>>> >>> works, then the invoker should not complain and the build  
>>>> should
>>>> >>> complete.
>>>> >>>
>>>> >>> A bunch more needs to be done... like creating a common test
>>>> >>> support class for selenium tests... and probably switching 

>>>> these
>>>> >>> tests to use testng, so that the selenium setup is done  
>>>> once, not
>>>> >>> for each test... as it does now... so expect Firefox to  
>>>> start and
>>>> >>> stop a few times.  We need to have some helpers to pick out
the
>>>> >>> console contents easily so they can be validated... I read that
>>>> >>> selenium supports xpath or something, but I have not actually
>>>> >>> tried any of that.
>>>> >>>
>>>> >>> This is just a very, very crude proof of concept... needs to
 
>>>> have
>>>> >>> reporting features added, capture logs, blah, blah, blah.
>>>> >>>
>>>> >>> Anyways, I committed this so we have a common base to work
>>>> >>> patches off of.
>>>> >>>
>>>> >>> One thing to note about the Selenium IDE, is that sometimes
it
>>>> >>> does not put into a waitForPageToLoad(), like if you record
a
>>>> >>> login to the console... you need to manually add that wait 

>>>> after
>>>> >>> the button is clicked or it will hose the test.
>>>> >>>
>>>> >>> I'm not married to any of this either... I just wanted  
>>>> something
>>>> >>> that worked... or kinda worked to show that its not going to
be
>>>> >>> that hard to make some rich integration tests.
>>>> >>>
>>>> >>> So, comments, suggestions, patches... all welcome.
>>>> >>>
>>>> >>> --jason
>>>> >>>
>>>> >>>
>>>> >>
>>>> >
>>>>
>>>>
>>>
>>
>


Mime
View raw message