flex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christofer Dutz <christofer.d...@c-ware.de>
Subject Re: [FlexJS] Selenium Webdriver Integrationtests
Date Thu, 19 Jan 2017 08:54:48 GMT
Ok, 

Today I finally got the geckodriver installed on the windows agent and I enabled the Selenium
tests in the test-suite.
Seems that they are correctly working ☺

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.flex.flexjs.examples.flexjsstore.FlexJSStoreIT
1484815122310	geckodriver	INFO	Listening on 127.0.0.1:2201
Jan 19, 2017 12:38:42 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Attempting bi-dialect session, assuming Postel's Law holds true on the remote end
1484815123195	mozprofile::profile	INFO	Using profile path C:\Users\jenkins\AppData\Local\Temp\1\rust_mozprofile.7M9qAGzvI4Oj
1484815123202	geckodriver::marionette	INFO	Starting browser C:\Program Files\Mozilla Firefox\firefox.exe
1484815123227	geckodriver::marionette	INFO	Connecting to Marionette on localhost:58549
1484815125382	Marionette	INFO	Listening on port 58549
Jan 19, 2017 12:38:47 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
JavaScript error: chrome://marionette/content/proxy.js, line 228: TypeError: l is undefined
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 12.934 sec - in org.apache.flex.flexjs.examples.flexjsstore.FlexJSStoreIT
Running org.apache.flex.flexjs.examples.helloworld.HelloWorldIT
1484815133631	geckodriver	INFO	Listening on 127.0.0.1:40511
Jan 19, 2017 12:38:54 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Attempting bi-dialect session, assuming Postel's Law holds true on the remote end
1484815134110	mozprofile::profile	INFO	Using profile path C:\Users\jenkins\AppData\Local\Temp\1\rust_mozprofile.puO8CEIgusxb
1484815134112	geckodriver::marionette	INFO	Starting browser C:\Program Files\Mozilla Firefox\firefox.exe
1484815134115	geckodriver::marionette	INFO	Connecting to Marionette on localhost:58585
1484815135170	Marionette	INFO	Listening on port 58585
Jan 19, 2017 12:38:56 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.381 sec - in org.apache.flex.flexjs.examples.helloworld.HelloWorldIT

Results :

Tests run: 3, Failures: 0, Errors: 0, Skipped: 1

So now it’s time to fill the test-suite with life ☺

Chris



Am 10.01.17, 22:48 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:

    Well I think I’ll write some test-that with Junit and try to test the examples a little.

    This should be able to detect problems in applications like with FlexJSStore, which doesn’t
seem to react on any button click or value-entering at all.
    I’m hoping on learning more about how to write tests for FlexJS and eventually even
how we could improve things to make them more easily testable.
    
    Chris
    
    Am 10.01.17, 18:02 schrieb "Alex Harui" <aharui@adobe.com>:
    
        In theory, we are in the business of write-once compile-to-many-targets,
        so, IMO, we should have a functional test harness as well as unit test
        harness that you write once to run on SWF and JS.
        
        But some folks aren't interested so much in SWF and are more interested in
        integration with existing JS-side test harnesses.
        
        I don't think we have to have just one test harness so folks interested in
        a particular approach can do what they need to get it working.  And maybe
        we can share some code, or maybe we can't.  No need to decide now, I just
        wanted to point out the assets we have in case it is helpful.
        
        -Alex
        
        On 1/10/17, 1:47 AM, "Yishay Weiss" <yishayjobs@hotmail.com> wrote:
        
        >Whichever path is taken I’d like to see more testing on the swf side. It
        >seems there’s little community interest now in making flash objects,
        >which is inline with the general public’s perception of flash as a thing
        >from the past. Still, my experience with working with FlexJS has been
        >that the flash workflow is still much faster than the HTML one. If we can
        >keep functional parity between flash and JS components, I think
        >developers would benefit significantly.
        >
        >
        >
        >
        >
        >From: Christofer Dutz<mailto:christofer.dutz@c-ware.de>
        >Sent: Tuesday, January 10, 2017 11:40 AM
        >To: dev@flex.apache.org<mailto:dev@flex.apache.org>
        >Subject: Re: [FlexJS] Selenium Webdriver Integrationtests
        >
        >
        >
        >Yeah exactly: Leave things the way they are with the old SDK, but build
        >up a proper unit-test and integration-test suite for FlexJS using
        >FlexUnit for Unit-Tests (located in the modules they test) and
        >Integration-Tests using Selenium (usually in dedicated testsuite modules).
        >
        >I would rather invest time in getting FlexUnit to work with FlexJS than
        >to spend time fixing Mustella … Was the next thing on my list after
        >finishing ASDoc static html generation … really have to continue working
        >on that :-(
        >
        >Chris
        >
        >Am 10.01.17, 10:31 schrieb "carlos.rovira@gmail.com im Auftrag von Carlos
        >Rovira" <carlos.rovira@gmail.com im Auftrag von
        >carlos.rovira@codeoscopic.com>:
        >
        >    Hi,
        >
        >    just to be sure we are talking about the same. The discussion is
        >about to
        >    left Mustella for old Flex SDK and go with Selenium for new FlexJS?
        >
        >    I did some work in the past fixing Mustella and I must to say that is
        >a
        >    real pain, but I'm ok to not spend more time on it and Flex SDK but
        >for
        >    fixing purposes.
        >
        >    Regarding new FlexJS, my vote is clear to go with Selenium. Although
        >I have
        >    little experience with that I think something that is more standard,
        >    integrates well with Maven and has a workflow that the community
        >likes, is
        >    a better way since we are making it from Zero.
        >
        >    Thanks Chris for working on this.
        >
        >    Carlos
        >
        >
        >    2017-01-09 23:22 GMT+01:00 Christofer Dutz
        ><christofer.dutz@c-ware.de>:
        >
        >    > Hi Alex,
        >    >
        >    > first of all I think we Need to distinguish between unit-tests and
        >    > functional-tests.
        >    >
        >    > For Unit-Tests I think FlexUnit should be the way to go. Ideally
        >runnable
        >    > in both SWF and JS. Here writing Tests in Flex(JS) is a good Thing.
        >    >
        >    > For Functional-Tests, pure frontend Tests are usually only one of
        >the
        >    > cases. I previously used JUnit-based Tests to Setup the backend and
        >run the
        >    > frontend Tests against a well prepared backend. Also do Frameworks
        >like
        >    > JUnit and even more TestNG provide great Support asynchron testing
        >and for
        >    > data-driven Tests. And I would like to mention that I used Tools
        >like
        >    > selenium to test Flex applications by utilizing the Flex Automation
        >API,
        >    > which worked like a charm. Ok … guess we would need something like
        >that.
        >    >
        >    > I would rather go down the path of well established and stable tools
        >    > instead of putting the burdon of learning yet another tool to
        >provide
        >    > functionality on people.
        >    >
        >    > Have any others had a look at the one Test I created? Would be
        >interested
        >    > in some more oppinions.
        >    >
        >    > Chris
        >    >
        >    >
        >    > Von meinem Samsung Galaxy Smartphone gesendet.
        >    >
        >    >
        >    > -------- Ursprüngliche Nachricht --------
        >    > Von: Alex Harui <aharui@adobe.com>
        >    > Datum: 09.01.17 19:47 (GMT+01:00)
        >    > An: dev@flex.apache.org
        >    > Betreff: Re: [FlexJS] Selenium Webdriver Integrationtests
        >    >
        >    > The goal of Mustella was to write tests in a declarative language,
        >in this
        >    > case, MXML.  That way, the test could be "interpreted" instead of
        >just
        >    > run, so the test harness could have control over timing and some
        >other
        >    > things that are really tricky in Flash like script timeouts and
        >needing to
        >    > allow the player to render.
        >    >
        >    > MXML would theoretically make tests shorter to write for the same
        >reasons
        >    > folks like declarative languages in general.  And reduce errors,
        >possibly
        >    > allow automated test generation, etc.
        >    >
        >    > So maybe the first question is:  Do we want to write tests that run
        >on
        >    > both SWF and JS?  I think we should since that reduces time to
        >create
        >    > tests for both platforms and potentially add some third platform in
        >the
        >    > future.  But if you are only interested in one platform, then sure,
        >go
        >    > ahead and leverage what is popular and available for that one
        >platform.
        >    >
        >    > And yes, it could be easier to take what is available for JS and
        >back port
        >    > it to SWF.  I was just trying to leverage what we (Apache and not
        >Adobe)
        >    > have in our repos.  There are a lot of existing tests for the
        >regular Flex
        >    > SDK that could potentially be used as is or with small
        >modifications.
        >    >
        >    > Is Mustella/Marmotinni buggy, poorly documented, in need of
        >improvement?
        >    > Yes.  It was never production-grade but was used to help us ship
        >Flex. I
        >    > just wanted to make sure you are aware it exists.
        >    >
        >    > -Alex
        >    >
        >    > On 1/8/17, 11:52 PM, "Christofer Dutz" <christofer.dutz@c-ware.de>
        >wrote:
        >    >
        >    > >Hi Alex,
        >    > >
        >    > >Well I have to admit that Mustella is a thing I always tried
        >avoiding as
        >    > >I never really managed to wrap my head around it. Just had a look
        >at the
        >    > >FlexJS version and it seems to be a copy of (parts) of Mustella
        >from the
        >    > >Flex SDK. At least from a few minutes of looking at the code I
        >couldn’t
        >    > >really understand the basic concepts of the tests.
        >    > >
        >    > >I can see some part which seems to be FlexJS code as well as some
        >Java
        >    > >code in a package “marmotinni” which is probably somehow related
        >to the
        >    > >neighbor directory “marmotinni”. I couldn’t really see any
        >documentation
        >    > >or code-comments that would explain things. And having a look at
        >the
        >    > >commit history for both directories, It’s only you contributing
to
        >this
        >    > >at all (except the initial commit of the marmotinni directory,
        >which was
        >    > >from eric). Also having a look at the history of the Flex SDK,
        >there
        >    > >seems to be a similar picture. It’s a very elite group of people
        >doing
        >    > >commits there and usually related to tweaking things instead of
        >really
        >    > >writing tests.
        >    > >
        >    > >If I would have to decide, I would definitely go the Junit +
        >Selenium
        >    > >path the way I set it up (But doesn’t have to be that way as long
        >as we
        >    > >stick to a more standard way that’s integratable into the Maven
        >build).
        >    > >The main reason for this is that it’s the way - I wouldn’t say
the
        >rest
        >    > >of the world, but the greater part of it – does things. It uses
        >well
        >    > >established mechanisms, which people are probably more familiar
        >with and
        >    > >definitely will be able to get more assistance, blogs, how-tos,
        >a.s.o.
        >    > >for. I know that setting up the initial project is challenging,
        >but I
        >    > >already did that for us. Now all we would need to do in order to
        >add new
        >    > >tests, is write a simple Junit test and eventually add a
        >dependency to a
        >    > >new example project.
        >    > >
        >    > >Another thing I don’t quite like with Mustella, the way I have
        >seen it
        >    > >with the old SDK, is that it doesn’t seem to be deterministic.
If
        >I need
        >    > >to loop tests several times in order for them to run, I can’t
        >trust the
        >    > >tool as a proper test tool.
        >    > >
        >    > >I would definitely vote for going down the more standard path than
        >the
        >    > >proprietary Adobe path for testing. Considering that we don’t
        >really have
        >    > >more than a proof of concept of a test-suite, I don’t think we
        >would be
        >    > >throwing away much.
        >    > >
        >    > >Chris
        >    > >
        >    > >
        >    > >Am 09.01.17, 06:37 schrieb "Alex Harui" <aharui@adobe.com>:
        >    > >
        >    > >    Did you look into how Mustella/Marmotinni works?  That is also
        >an
        >    > >attempt
        >    > >    to use Selenium.
        >    > >
        >    > >    -Alex
        >    > >
        >    > >    On 1/8/17, 11:23 AM, "Christofer Dutz"
        ><christofer.dutz@c-ware.de>
        >    > >wrote:
        >    > >
        >    > >    >Hi,
        >    > >    >
        >    > >    >After noticing that some examples seem to compile fine, but
        >don’t
        >    > >    >actually work, I decided to invest a day in building
        >integration
        >    > >tests to
        >    > >    >test the examples in a browser (currently Firefox).
        >    > >    >As this requires the “Geckodriver” to be installed, I
made
        >the tests
        >    > >auto
        >    > >    >activate themselves as soon as the “webdriver.gecko.driver“
        >property
        >    > >is
        >    > >    >set to the path of the geckodriver executable.
        >    > >    >
        >    > >    >What happens is:
        >    > >    >
        >    > >    >-          The tests are compiled
        >    > >    >
        >    > >    >-          A Tomcat8 is setup and started
        >    > >    >
        >    > >    >-          the HelloWorld example is deployed
        >    > >    >
        >    > >    >-          The Selenium JUnit Tests are run
        >    > >    >
        >    > >    >-          The tomcat is shutdown
        >    > >    >
        >    > >    >Please have a look at this, if you think this is a valid
        >path, then I
        >    > >    >would start writing some test-cases that basically test some
        >of our
        >    > >    >examples.
        >    > >    >
        >    > >    >Get geckodriver from here:
        >    > >https://github.com/mozilla/geckodriver/releases
        >    > >    >I needed to update my Firefox to the latest version in order
        >for the
        >    > >    >tests to run.
        >    > >    >
        >    > >    >Chris
        >    > >
        >    > >
        >    > >
        >    >
        >    >
        >
        >
        >    --
        >
        >    Carlos Rovira
        >    Director General
        >    M: +34 607 22 60 05
        >    http://www.codeoscopic.com
        >    http://www.avant2.es
        >
        >    Este mensaje se dirige exclusivamente a su destinatario y puede
        >contener
        >    información privilegiada o confidencial. Si ha recibido este mensaje
        >por
        >    error, le rogamos que nos lo comunique inmediatamente por esta misma
        >vía y
        >    proceda a su destrucción.
        >
        >    De la vigente Ley Orgánica de Protección de Datos (15/1999), le
        >comunicamos
        >    que sus datos forman parte de un fichero cuyo responsable es
        >CODEOSCOPIC
        >    S.A. La finalidad de dicho tratamiento es facilitar la prestación del
        >    servicio o información solicitados, teniendo usted derecho de acceso,
        >    rectificación, cancelación y oposición de sus datos dirigiéndose a
        >nuestras
        >    oficinas c/ Paseo de la Habana 9-11, 28036, Madrid con la
        >documentación
        >    necesaria.
        >
        >
        
        
    
    

Mime
View raw message