geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Dillon <ja...@planet57.com>
Subject Re: Testsuite module, with basic/crude Selenium support
Date Sun, 03 Sep 2006 11:37:30 GMT
I think I'm going to make 2 test groups for now... one to test testng  
and one to test the junit decorator bits... then we can see which  
works better.

I've been playing with console-testsuite today using testng and it  
works well... nice to have tags to specify what methods to run before  
the suite and after the suite... I think we might have to do some  
magic with JUnit to make that work.

I also hooked up selenium to be able to use some custom extensions...  
specifically the ability to removeCookie(), but we can add more as  
needed.

And, I changed the way that the server starting goals work, so that  
you can run them on the command-line with maven to start them up,  
which is much better to help develop.  For example you can:

     mvn selenium:start &
     mvn geronimo:start &
     ...
     mvn geronimo:stop

This will default to starting up Jetty + J2EE, but if you want to  
test with Tomcat + J2EE, you can:

     mvn geronimo:start -DassemblyId=tomcat
     ...
     mvn geronimo:stop -DassemblyId=tomcat

jetty-minimal and tomcat-minimal can also be used with artifactId,  
though until I get a better way to tell when the server is fully  
started via JMX, background mode for these assemblies won't work.

They can still be background'd for testsuite usage, though needs to  
have the background flag set to true.

The geronimo:start goal is also smart enough to detect when a new  
assembly is available, so the next time you run it it will re- 
install... else it will just reuse the previously extracted assembly.

Probably need to expose a command-line param to allow the basedir to  
be changed... for windows folks... since the default is to unzip  
under console-testsuite/target/, which I'd guess is probably going to  
be a tad to long for windows.

Plugin docs have been updated if anyone is interested:

     http://geronimo.apache.org/maven/server/maven-plugins/geronimo- 
maven-plugin/index.html
     http://geronimo.apache.org/maven/server/maven-plugins/selenium- 
maven-plugin/index.html

Finally... geronimo:start and stop are usable from the main tree as  
well, which makes it nice and easy to build and run the server:

     mvn install
     mvn geronimo:start -Dverbose=true

  * * *

So I'm going to svn cp console-testsuite/basic to console-testsuite/ 
basic-junit and then apply your patch to that tree, and then commit  
the changes to basic that I have done to use testng, plus some other  
support/helper classes.

Oh, and I did get some xpath validation working... but the xpath  
string is horrible... I used XPather (http://xpath.alephzarro.com/)  
to select some text and give me the xpath.  It did work but ends up  
like:

<snip>
assertEquals("Server Info", selenium.getText(
       "xpath=/html/body/table[@id='rootfragment']/tbody/tr[2]/td/ 
table/tbody/tr[2]/td[4]/table/tbody/tr[1]/td/table/tbody/tr[1]/td/div/ 
table/tbody/tr/td[2]/table/tbody/tr/td[1]/strong"));
</snip>

We may need to insert some id's into the console's pages to allow us  
to root to the major panels.

--jason


On Sep 2, 2006, at 6:28 AM, Bill Dudney wrote:

> Hi Jason,
>
> This is very cool indeed, thanks for putting the prototype  
> together. I just submitted a patch that makes firefox pop up only  
> once to run all the tests (it actually pops up twice, once during  
> selenium.start() and then once for the test suite but the first  
> time is for a split second).
>
> http://issues.apache.org/jira/browse/GERONIMO-2374
>
> It uses a JUnit TestDecorator to run setup only once for the whole  
> suite.
>
> IMO until we move to JDK 5 its not a great idea to move to TestNG,  
> you will get some benefits for sure but the source has to be parsed  
> at runtime to get at the meta-data (i.e. the info about the tests).  
> Krufty IMO.
>
> The patch has a comment about invalid XHTML.  I believe the invalid  
> XHTML part of the console is preventing the XPath find from working.
>
> TTFN,
>
> -bd-
>
>
> On Sep 1, 2006, at 7:46 PM, Jason Dillon wrote:
>
>> I've checked in a new top-level testsuite module and a few new  
>> plugins to support it.  This is only the start and I expect it to  
>> change over the next few weeks (er maybe months) as momentum  
>> starts to pick up.
>>
>> I looked into Cargo, and while I think we should eventually use it  
>> to start/stop the server... asis now its broke.  We need to  
>> provide a very simple and consistent way for Cargo to invoke some  
>> operations via JMX, which I hinted to in previous emails.  Once  
>> that is done, we can patch Cargo and hopefully get that committed  
>> to support G... but in the mean time I whipped up a simple start &  
>> stop mojo that uses Ant to start and stop the server.  Its very,  
>> very crude and we need to fix stuff like logging output etc.
>>
>> I also played around a little with Selenium to make some tests for  
>> the console.  It wants a special server process started, so I  
>> created a selenium plugin which currently only starts that server  
>> in... well an icky way, but should work for the moment.  Need to  
>> ask the m2 folks how to do this better.
>>
>> I created a console-testsuite module, which sets up the G server  
>> and Selenium server in pre-integration-test, and then uses the  
>> maven-invoker-plugin to invoke the basic module to actually run  
>> the tests.  Only one test class right now, SimpleLoginTest, which  
>> does just that... logs in, logs in and then out and then another  
>> that logs in and clicks some links.
>>
>> I've got it all running from one mvn command and so far, on my  
>> mac, it works fine.  I'd imagine due to the location of the  
>> assembly that Windows will not be able to run this... we'll need  
>> to fix this later.  But those of you lucky enough to have a Linux  
>> box or a mac, you should be able to:
>>
>>     mvn clean && mvn install
>>     cd testsuite
>>     mvn
>>
>> (the testsuite is not hooked up to the default build)
>>
>> And you should see Firefox popup a few times as Selenium goes  
>> about clicking on stuff.  Right now its only really doing  
>> validation of the console page's titles... but if everything  
>> works, then the invoker should not complain and the build should  
>> complete.
>>
>> A bunch more needs to be done... like creating a common test  
>> support class for selenium tests... and probably switching these  
>> tests to use testng, so that the selenium setup is done once, not  
>> for each test... as it does now... so expect Firefox to start and  
>> stop a few times.  We need to have some helpers to pick out the  
>> console contents easily so they can be validated... I read that  
>> selenium supports xpath or something, but I have not actually  
>> tried any of that.
>>
>> This is just a very, very crude proof of concept... needs to have  
>> reporting features added, capture logs, blah, blah, blah.
>>
>> Anyways, I committed this so we have a common base to work patches  
>> off of.
>>
>> One thing to note about the Selenium IDE, is that sometimes it  
>> does not put into a waitForPageToLoad(), like if you record a  
>> login to the console... you need to manually add that wait after  
>> the button is clicked or it will hose the test.
>>
>> I'm not married to any of this either... I just wanted something  
>> that worked... or kinda worked to show that its not going to be  
>> that hard to make some rich integration tests.
>>
>> So, comments, suggestions, patches... all welcome.
>>
>> --jason
>>
>>
>


Mime
View raw message