royale-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harbs <harbs.li...@gmail.com>
Subject Re: Unit Tests et. al.
Date Mon, 06 Nov 2017 09:42:05 GMT
Well, Josh reversed the order of the arguments, and I think that makes sense. The string is
optional and it was at the beginning. It’s now the optional last argument and all the arguments
are as typed as they can be. (compare arguments are of type Object.)

I just merged the testing branch into develop, so tests can be added on the develop branch.
The js testing is not functional yet, but the tests can be written in a platform agnostic
way.

More to come…

Harbs

> On Nov 6, 2017, at 12:51 AM, Piotr Zarzycki <piotrzarzycki21@gmail.com> wrote:
> 
> My guess is that, cause this approach emulates JUnit implementation. Let's
> say assertFalse - you can pass there one argument Boolean or two arguments
> Boolean and message. In AS3 there is no ability to create two functions
> with the same name. But in Java method overloading probably exists, so in
> JUnit there is probably:
> 
> assertFalse(Boolean param);
> assertFalse(Boolean param, String);
> 
> Summarize everything - I see only some AS3 language restriction as an
> reason.
> 
> Piotr
> 
> 
> 
> 2017-11-05 23:14 GMT+01:00 Harbs <harbs.lists@gmail.com>:
> 
>> BTW:
>> 
>> Was there a good reason FlexUnit uses the spread operator for assert
>> arguments instead of typed arguments?
>> 
>>> On Nov 5, 2017, at 11:36 PM, Harbs <harbs.lists@gmail.com> wrote:
>>> 
>>> Some progress on the unit tests:
>>> 
>>> I got the Core tests to run on the swf side when doing a full build. In
>> order to do that I needed to move the running of the tests until after
>> Basic finished building so the test app could be built.
>>> 
>>> We’re going to need to do something similar with the Maven build.
>>> 
>>> The Testing project is compiling and is usable (on a basic level) too.
>> Tests should run with both a SWF and JS target. The SWF target uses
>> FlexUnit, while the JS target uses Josh’s implementation.
>>> 
>>> I also migrated the Core tests to use the new project and that is
>> building as well.
>>> 
>>> I still need to add the bits that Greg wrote and wire up some JS tests.
>>> 
>>> Hopefully I’ll make some more progress tomorrow.
>>> 
>>> Harbs
>>> 
>>>> On Nov 5, 2017, at 8:17 AM, Alex Harui <aharui@adobe.com.INVALID>
>> wrote:
>>>> 
>>>> Those warnings do not seem to affect the success of the tests on SWF for
>>>> me.
>>>> 
>>>> IMO, the big challenge is in capturing test results so it can be
>> reported
>>>> in a CI dashboard.  Having a GUI display results in Flash or in a
>> browser
>>>> is a great first step, but I believe going from there to getting
>> Selenium
>>>> to report individual test failures may be trickier.
>>>> 
>>>> Peter tried to write up Mustella a long time ago. It might need
>> updating.
>>>> See:
>>>> https://cwiki.apache.org/confluence/display/FLEX/Mustella+Overview
>>>> 
>>>> The advantage of declarative testing languages is that you get to
>>>> constrain the kinds of things that a test can do, and abstract when the
>>>> test harness does it.  It is much harder to interrupt sequential lines
>> of
>>>> ActionScript/JavaScript.
>>>> 
>>>> HTH,
>>>> -Alex
>>>> 
>>>> On 11/4/17, 3:13 PM, "Harbs" <harbs.lists@gmail.com> wrote:
>>>> 
>>>>> It looks like FlexUnit is currently broken.
>>>>> 
>>>>> I’m getting the following output when trying to run the FlexUnit tests
>> on
>>>>> Core and Basic:
>>>>> 
>>>>> [mxmlc]
>>>>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/
>> FlexUnit4/target/flexuni
>>>>> t-4.3.0-20140410-as3_4.12.0.swc Warning: The definition
>>>>> mx.utils.ObjectUtil depended on by org.flexunit.runner.Description in
>> the
>>>>> SWC
>>>>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/
>> FlexUnit4/target/flexuni
>>>>> t-4.3.0-20140410-as3_4.12.0.swc could not be found
>>>>> [mxmlc]
>>>>> [mxmlc]
>>>>> [mxmlc]
>>>>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/
>> FlexUnit4/target/flexuni
>>>>> t-4.3.0-20140410-as3_4.12.0.swc Warning: The definition
>>>>> mx.utils.StringUtil depended on by flexunit.framework.AsyncTestHelper
>> in
>>>>> the SWC
>>>>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/
>> FlexUnit4/target/flexuni
>>>>> t-4.3.0-20140410-as3_4.12.0.swc could not be found
>>>>> 
>>>>> Has anyone looked at this yet?
>>>>> 
>>>>> Harbs
>>>>> 
>>>>>> On Nov 4, 2017, at 10:45 PM, Harbs <harbs.lists@gmail.com>
wrote:
>>>>>> 
>>>>>> Thanks! That looks very useful.
>>>>>> 
>>>>>> I started work on a feature/testing branch. I’ll try to merge what
you
>>>>>> did into what Josh did. I’m going to try to get the existing Core
>> and/or
>>>>>> Basic tests working in both swf and node js output tomorrow. We’ll
see
>>>>>> how well I do… ;-)
>>>>>> 
>>>>>> We might need different configs for different testing environments,
>> but
>>>>>> I’ll see if we can keep the divergence of the environs is minimal
as
>>>>>> possible.
>>>>>> 
>>>>>> Once I get the basics worked out, I’ll likely have an idea what
others
>>>>>> can help with. Let’s see if we can whip the Royale tests into shape.
>> :-)
>>>>>> 
>>>>>> Harbs
>>>>>> 
>>>>>>> On Nov 4, 2017, at 10:30 PM, Greg Dove <greg.dove@gmail.com>
wrote:
>>>>>>> 
>>>>>>> Harbs, I don't know if it still works since changes to Royale,
but I
>>>>>>> had
>>>>>>> something rudimentary for cross-target unit testing quite shortly
>> after
>>>>>>> working on the reflection support last year, in fact that was
my
>>>>>>> primary
>>>>>>> reason for choosing to focus on reflection at the time.
>>>>>>> 
>>>>>>> It was visual output only (i.e. just to look at the test results
>>>>>>> output in
>>>>>>> the browser) and the goal was to get some compatibility with
flexunit
>>>>>>> (so
>>>>>>> that the same swf-based flexunit tests that were running in the
build
>>>>>>> -e..g
>>>>>>> BinaryData tests- could be run visually side by side between
swf and
>>>>>>> js).
>>>>>>> I also added a new TestVariance metadata to account for known
>>>>>>> (expected?)
>>>>>>> variation between swf and js. This was needed to cover (for example)
>>>>>>> things
>>>>>>> like testing Reflection classes themselves, where the results
can be
>>>>>>> different between the targets based on the 'native' base classes
>> (e..g
>>>>>>> EventDispatcher inheritance chain).
>>>>>>> 
>>>>>>> Some of that code might also be useful to harvest for what you
are
>>>>>>> doing,
>>>>>>> I'm not sure...
>>>>>>> 
>>>>>>> It's here:
>>>>>>> 
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fgithub.
>>>>>>> com%2Fapache%2Froyale-asjs%2Ftree%2Fdevelop%
>> 2Fmanualtests%2FUnitTests&da
>>>>>>> ta=02%7C01%7C%7C67a58a332b6942a2022108d523d1
>> 5d35%7Cfa7b1b5a7b34438794aed
>>>>>>> 2c178decee1%7C0%7C0%7C636454304479378270&sdata=
>> DX9gXiLqUoFQizp6xrakUwxjl
>>>>>>> cdyHMlZO5mOzDSiKDQ%3D&reserved=0
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Sun, Nov 5, 2017 at 7:30 AM, Harbs <harbs.lists@gmail.com>
wrote:
>>>>>>> 
>>>>>>>> Awesome!
>>>>>>>> 
>>>>>>>> I assume that this means I can code the code to the Royale
repo
>> under
>>>>>>>> Apache 2. Right?
>>>>>>>> 
>>>>>>>> Harbs
>>>>>>>> 
>>>>>>>>> On Nov 3, 2017, at 5:37 PM, Josh Tynjala <joshtynjala@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Please feel free to use my test runner. I'm happy to
commit it
>>>>>>>>> somewhere
>>>>>>>> to make it official Apache code, if necessary.
>>>>>>>>> 
>>>>>>>>> - Josh
>>>>>>>>> 
>>>>>>>>> On 2017-11-03 04:19, Harbs <harbs.lists@gmail.com>
wrote:
>>>>>>>>>> One topic which keeps coming up is better test coverage
for
>> Royale.
>>>>>>>>>> 
>>>>>>>>>> I think this is becoming a critical issue for a few
reasons:
>>>>>>>>>> 1. As we get close to version 1.0 it’s necessary
to have good
>> test
>>>>>>>> coverage for confidence of quality and confidence that we
don’t
>>>>>>>> introduce
>>>>>>>> recessive bugs.
>>>>>>>>>> 2. It’s really hard to accept Github pull
requests without
>>>>>>>>>> examining
>>>>>>>> the code VERY well that it does not introduce recessive bugs.
CI
>>>>>>>> which runs
>>>>>>>> automated tests could give a preliminary test on pull requests
to
>>>>>>>> ensure
>>>>>>>> that they don’t break anything. If the pull requests
do break
>>>>>>>> something,
>>>>>>>> it allows the requester to fix the problem with confidence
without
>>>>>>>> taking
>>>>>>>> others’ time.
>>>>>>>>>> 
>>>>>>>>>> I think we need to break up testing into pieces and
figure out a
>>>>>>>> strategy to implement automated tests in a way that they
are
>>>>>>>> maintainable.
>>>>>>>>>> 
>>>>>>>>>> Some points:
>>>>>>>>>> 1. I think integration into something like Travis
would be very
>>>>>>>>>> helpful.
>>>>>>>>>> 2. I think there’s a Jenkins plugin for building
pull requests.
>>>>>>>>>> Not
>>>>>>>> sure how useful it is.[1]
>>>>>>>>>> 3. Josh has created a Node.js-compatible test-runner
architecture
>>>>>>>>>> which
>>>>>>>> could be useful for unit tests on parts of the framework
which
>>>>>>>> don’t rely
>>>>>>>> on browser features. (i.e. models and the like) He mentioned
the
>>>>>>>> possibility of donating it, and I think that would be a very
useful
>>>>>>>> feature.
>>>>>>>>>> 4. For UI and integration tests, there seem to be
some pretty cool
>>>>>>>> integrations using Selenium.[2][3]
>>>>>>>>>> 5. I think the main testing effort needs to be using
JS and
>>>>>>>>>> something
>>>>>>>> like Josh’s testing framework for non-UI pieces and
some
>> easy-to-use
>>>>>>>> Selenium integration for visual pieces and integration tests.
>>>>>>>>>> 6. We probably also want some API endpoints we can
test off of for
>>>>>>>> integration testing.
>>>>>>>>>> 
>>>>>>>>>> I’m willing to invest time into this.
>>>>>>>>>> 
>>>>>>>>>> It’s going to be a lot of work building out
the tests and I
>> think
>>>>>>>>>> we
>>>>>>>> need a plan for that. My thoughts:
>>>>>>>>>> 
>>>>>>>>>> 1. Step one is to make it easy to write meaningful
automated tests
>>>>>>>>>> and
>>>>>>>> establish a clear pattern.
>>>>>>>>>> 2. Step two is to start writing tests starting from
the
>>>>>>>> most-used/easiest to beak pieces and work out from there.
>>>>>>>>>> 3. Once the pattern is established, any new pieces
MUST have
>> testing
>>>>>>>> coverage.
>>>>>>>>>> 4. When fixing bugs, attention should be paid to
adding testing
>> for
>>>>>>>> that component.
>>>>>>>>>> 5. When a pull request comes in on a piece which
does not have
>> unit
>>>>>>>> test, a test must be written before accepting the pull request.
The
>>>>>>>> test
>>>>>>>> does not need to be written by the requester. Before examining
the
>>>>>>>> request,
>>>>>>>> the test should be written to pass for expected behavior
and fail
>> for
>>>>>>>> the
>>>>>>>> bug that the pull request is attempting to fix (assuming
the pull
>>>>>>>> request
>>>>>>>> is to fix a bug).
>>>>>>>>>> 
>>>>>>>>>> Thoughts?
>>>>>>>>>> Harbs
>>>>>>>>>> 
>>>>>>>>>> P.S. I’m thinking of coming to the US in late
December/early
>>>>>>>>>> January.
>>>>>>>> I would be interested in getting together for a hacking session
with
>>>>>>>> folks
>>>>>>>> who are available.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> [1]https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fw
>>>>>>>>>> iki.jenkins.io%2Fdisplay%2FJENKINS%2FGitHub%2Bpull&
>> data=02%7C01%7C%7C
>>>>>>>>>> 67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178de
>> cee1%7
>>>>>>>>>> C0%7C0%7C636454304479378270&sdata=cnyBtnVycpg3Aa7Hfel3dkAlez2m7M
>> 0uSl3
>>>>>>>>>> f0ezbSZY%3D&reserved=0+
>>>>>>>> request+builder+plugin
>>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fwiki.
>>>>>>>> jenkins.io%2F&data=02%7C01%7C%7C67a58a332b6942a2022108d523d1
>> 5d35%7Cfa7b
>>>>>>>> 1b5a7b34438794aed2c178decee1%7C0%7C0%7C636454304479378270&
>> sdata=eqe6ec%
>>>>>>>> 2F%2FIatSxdUuHU0Lo8o3WxySBkdSvUNa6i9NQ%2FI%3D&reserved=0
>>>>>>>> display/JENKINS/GitHub+pull+request+builder+plugin>
>>>>>>>>>> 
>>>>>>>>>> [2]https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fd
>>>>>>>>>> ocs.travis-ci.com%2Fuser%2Fgui-and-headless-browsers%
>> 2F&data=02%7C01%
>>>>>>>>>> 7C%7C67a58a332b6942a2022108d523d15d35%
>> 7Cfa7b1b5a7b34438794aed2c178dec
>>>>>>>>>> ee1%7C0%7C0%7C636454304479378270&sdata=zK%
>> 2FdOBmWJUsF7SylIQtMQQpAeOhw
>>>>>>>>>> 03sjOytutB4rGCY%3D&reserved=0 <
>>>>>>>> 
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fdocs.t
>>>>>>>> ravis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%
>> 7C01%7C%7C67
>>>>>>>> a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178de
>> cee1%7C0%7
>>>>>>>> C0%7C636454304479378270&sdata=zK%2FdOBmWJUsF7SylIQtMQQpAeOhw03s
>> jOytutB4
>>>>>>>> rGCY%3D&reserved=0>
>>>>>>>>>> 
>>>>>>>>>> [3]https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fs
>>>>>>>>>> aucelabs.com%2Fproducts%2Fintegrations&data=02%7C01%
>> 7C%7C67a58a332b69
>>>>>>>>>> 42a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178de
>> cee1%7C0%7C0%7C636
>>>>>>>>>> 454304479378270&sdata=kwfto9xiP0ygeBrEqGz9ucdC%
>> 2FNXSRVOQho9qUW1mqdw%3
>>>>>>>>>> D&reserved=0
>>>>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fsau
>>>>>>>>>> celabs.com%2F&data=02%7C01%7C%7C67a58a332b6942a2022108d523d1
>> 5d35%7Cfa
>>>>>>>>>> 7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636454304479378270&
>> sdata=xpG
>>>>>>>>>> G85mRtcFwwG7nbRfT7oLMPm4VPeHxe4trkG1wrAw%3D&reserved=0
>>>>>>>> products/integrations>
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>> 
> 
> 
> -- 
> 
> Piotr Zarzycki
> 
> Patreon: *https://www.patreon.com/piotrzarzycki
> <https://www.patreon.com/piotrzarzycki>*


Mime
View raw message