royale-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Tynjala"<joshtynj...@apache.org>
Subject Re: Unit Tests et. al.
Date Fri, 03 Nov 2017 15:37:03 GMT
Please feel free to use my test runner. I'm happy to commit it somewhere to make it official
Apache code, if necessary.

- Josh

On 2017-11-03 04:19, Harbs <harbs.lists@gmail.com> wrote: 
> One topic which keeps coming up is better test coverage for Royale.
> 
> I think this is becoming a critical issue for a few reasons:
> 1. As we get close to version 1.0 it’s necessary to have good test coverage for
confidence of quality and confidence that we don’t introduce recessive bugs.
> 2. It’s really hard to accept Github pull requests without examining the code VERY
well that it does not introduce recessive bugs. CI which runs automated tests could give a
preliminary test on pull requests to ensure that they don’t break anything. If the pull
requests do break something, it allows the requester to fix the problem with confidence without
taking others’ time.
> 
> I think we need to break up testing into pieces and figure out a strategy to implement
automated tests in a way that they are maintainable.
> 
> Some points:
> 1. I think integration into something like Travis would be very helpful.
> 2. I think there’s a Jenkins plugin for building pull requests. Not sure how useful
it is.[1]
> 3. Josh has created a Node.js-compatible test-runner architecture which could be useful
for unit tests on parts of the framework which don’t rely on browser features. (i.e.
models and the like) He mentioned the possibility of donating it, and I think that would be
a very useful feature.
> 4. For UI and integration tests, there seem to be some pretty cool integrations using
Selenium.[2][3]
> 5. I think the main testing effort needs to be using JS and something like Josh’s
testing framework for non-UI pieces and some easy-to-use Selenium integration for visual pieces
and integration tests.
> 6. We probably also want some API endpoints we can test off of for integration testing.
> 
> I’m willing to invest time into this.
> 
> It’s going to be a lot of work building out the tests and I think we need a plan
for that. My thoughts:
> 
> 1. Step one is to make it easy to write meaningful automated tests and establish a clear
pattern.
> 2. Step two is to start writing tests starting from the most-used/easiest to beak pieces
and work out from there.
> 3. Once the pattern is established, any new pieces MUST have testing coverage.
> 4. When fixing bugs, attention should be paid to adding testing for that component.
> 5. When a pull request comes in on a piece which does not have unit test, a test must
be written before accepting the pull request. The test does not need to be written by the
requester. Before examining the request, the test should be written to pass for expected behavior
and fail for the bug that the pull request is attempting to fix (assuming the pull request
is to fix a bug).
> 
> Thoughts?
> Harbs
> 
> P.S. I’m thinking of coming to the US in late December/early January. I would be
interested in getting together for a hacking session with folks who are available.
> 
> [1]https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin <https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin>
> [2]https://docs.travis-ci.com/user/gui-and-headless-browsers/ <https://docs.travis-ci.com/user/gui-and-headless-browsers/>
> [3]https://saucelabs.com/products/integrations <https://saucelabs.com/products/integrations>

Mime
View raw message