cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Grieve <agri...@chromium.org>
Subject Re: mobile-spec and releases: How do we test?
Date Sat, 12 Oct 2013 01:08:34 GMT
The eval of the jasmine interface deserves mention. Is the motivation there
that tests can choose to use another testing framework? That's why you
don't just make jasmine functions globals?

One nit pick just from reading your email is that this will cause the test
js-modules to be injected into apps that use the plugins. I think probably
we will want to update the tools to recognize a <js-test-module>. We
*could* work around it by adding the tests to new plugins that depend on
the thing they are testing, but I think changing the tools would be nicer.

Another nit is that it would be nice if the CordovaTests app didn't depend
on any plugins. e.g., don't have it depend on device plugin.

Overall, really like the approach!


On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny <mmocny@chromium.org> wrote:

> TLDR; I've implemented a plugin testing strategy that requires 0 new
> tooling features, can support non-core plugins, and hopefully even support
> a variety of methods for running/reporting the test results.  Super alpha
> preview, but take a look if you like the direction!
>
>
> NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
>
> NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
>
> UPDATED: Converted three existing plugins to this "new style".  Code is on
> feature branches of respective repos (cdvtest branch):
> org.apache.cordova.device
> org.apache.cordova.device-motion
> org.chromium.storage
> (must clone locally, switch to branch, and plugin add from local path)
>
>
> Now, any plugin that wants to join in on the fun needs to provide a
> <js-module
> src="..." name="test" /> and use this template:
>
> exports.init = function() {
>   eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
> 'this'));
>
>   // TESTS GO HERE
>   describe(..., function() {
>     it(...);
>   });
> };
> (The eval is optional but super useful; it will inject the jasmine
> interface into your scope, so you don't have to type jasmine.describe,
> jasmine.it, etc.  Not sure of any cleaner way to do this.)
>
>
> STEPS:
> 1. create a new cordova project
> 2. import the CordovaTest app into your www
> 3. add any or all of the above plugins
> 4. give it a run, and try out the auto tests (all pass on ipad, some still
> fail on android)
>
> Lots of work left to do, but hopefully good enough to whet your appetite.
>
> One thing to note, I've attempted to make CDVTest and plugin tests unaware
> of the specific jasmine version the app is hosting (so that it can be
> swapped without changing all plugins), but it must support the new style
> interface for async tests (ie, accept a done callback).  This is the style
> that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use (I'm
> using jasmine 2.0 rc3).  This means that core plugin tests require some
> code transformations, but the net effect is cleaner tests and more common
> style with our node tools' tests.
>
> Manual tests are not implemented yet.
>
> -Michal
>
> On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny <mmocny@chromium.org>
> wrote:
>
> > I'm throwing something together right now, actually.  I'll post my
> current
> > progress today so you can take a look.
> >
> >
> > On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux <b@brian.io> wrote:
> >
> >> Sorry keep meaning to respond. I like Michal's first step but growing
> to a
> >> full suite of tools. Are you currently tackling this Braden? I feel like
> >> it
> >> is related to the Medic stuff and maybe we should throw one of our guys
> on
> >> the problem fully.
> >>
> >>
> >> On Sep 27, 2013 5:10 PM, "Braden Shepherdson" <braden@chromium.org>
> >> wrote:
> >>
> >> > Which one?
> >> >
> >> >
> >> > On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux <b@brian.io> wrote:
> >> >
> >> > > I really like your proposal as a starting point. Very simple but
> would
> >> > > allow for in-app testing as well as on the cmd line if we so wish.
> >> > >
> >> > >
> >> > > On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny <mmocny@chromium.org>
> >> > wrote:
> >> > >
> >> > > > I was looking over some old emails from this list on plugin
> testing,
> >> > and
> >> > > an
> >> > > > idea that was proposed way back was to ship plugin tests as a
> second
> >> > > > plugin.  That way, you can chose to install tests, or not, and
> know
> >> > > > explicitly if they are being copied into your final project.
> >> > > >
> >> > > > An alternative would be to support build targets a la
> >> "release/debug"
> >> > and
> >> > > > have target-specific plugin.xml tags (assets, js-modules,
> >> > source-file..).
> >> > > >
> >> > > > -Michal
> >> > > >
> >> > > >
> >> > > > On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux <b@brian.io>
wrote:
> >> > > >
> >> > > > > I think this is basically what we've been proposing for
a while
> >> now.
> >> > > > >
> >> > > > >
> >> > > > > On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny <
> >> mmocny@chromium.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > > I would suggest perhaps a simpler approach, which doesn't
add
> >> > > anything
> >> > > > > new
> >> > > > > > to cordova-cli/plugman:
> >> > > > > >
> >> > > > > > - Each plugin ships with a "tests" js-module, and we
document
> a
> >> > > > > convention
> >> > > > > > of where they should live, and what signature it should
have
> >> (i.e.,
> >> > > > > > cordova.require('plugin.name.Tests').forEach(...) ).
> >> > > > > >   - Will need a common way to describe/report results
(others
> >> have
> >> > > > > > mentioned TAP).
> >> > > > > > - Any app is free to run those plugin tests in any
which way,
> >> but
> >> > we
> >> > > > > ship a
> >> > > > > > mobile-spec app which is one opinionated way to do
so.
> >> > > > > >   - It attempts to require the test module for each
installed
> >> > plugin,
> >> > > > > runs
> >> > > > > > them, and aggregates results.
> >> > > > > >   - It could report results to some shared server,
allow
> >> toggling
> >> > of
> >> > > > > tests,
> >> > > > > > etc, but no plugin should know or care about those
features.
> >> > > > > >
> >> > > > > > Using that as a generic base:
> >> > > > > >
> >> > > > > > - We ship a "CDVTests" (or whatever) plugin which has
a bunch
> of
> >> > > > library
> >> > > > > > code for creating tests, and plugins can use it to
register
> >> their
> >> > > > tests.
> >> > > > > > - This makes it easier to register manual tests in
a common
> >> format
> >> > > for
> >> > > > > core
> >> > > > > > plugins, and prevents code duplication for core auto
tests.
> >> > > > > > - External plugins can chose to use our testing library,
or
> not.
> >> > > > > >
> >> > > > > > -Michal
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
<
> >> > > > > braden@chromium.org
> >> > > > > > >wrote:
> >> > > > > >
> >> > > > > > > Here's an off-the-top-of-my-head sketch of how
we might do
> >> > Voltron
> >> > > > > tests:
> >> > > > > > >
> >> > > > > > > - Add a tag to plugin.xml that names each test
file:
> >> > > > > > >     <test type="automatic" src="spec/foo.js"
name="Foo
> >> Automated"
> >> > > />
> >> > > > > > >     <test type="manual" src="spec/bar.js" name="Foo
Manual"
> />
> >> > > > > > > - Add a new command, cordova test (maybe prepare-test),
> that:
> >> > > > > > >     - Ignores the top-level www.
> >> > > > > > >     - Instead copies in a basic testing index.html
similar
> to
> >> the
> >> > > > > current
> >> > > > > > > mobile-spec's
> >> > > > > > >     - That index reads a file akin to cordova_plugins.js
> >> > > > > > (cordova_tests.js,
> >> > > > > > > maybe?) generated by the CLI, containing the info
from the
> >> <test>
> >> > > > tags.
> >> > > > > > >     - It has navigation similar to the current
mobile-spec,
> >> with
> >> > > > > buttons
> >> > > > > > > for the automatic and manual sections. Auto has
"All" and
> then
> >> > each
> >> > > > > > module,
> >> > > > > > > manual just has the list of modules.
> >> > > > > > >
> >> > > > > > > Thoughts?
> >> > > > > > >
> >> > > > > > > Braden
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana
<
> >> > > > csantana23@gmail.com
> >> > > > > > > >wrote:
> >> > > > > > >
> >> > > > > > > > I like the idea can we call mobilespec now
cordova-voltron
> >> and
> >> > be
> >> > > > DRY
> >> > > > > > and
> >> > > > > > > > use the tests form the plugins.
> >> > > > > > > >
> >> > > > > > > > Voltron by itself creates an App that tests
only core, but
> >> as
> >> > you
> >> > > > > > > > use plugman to add plugins to voltron it
has more test
> >> cases.
> >> > > > > > > >
> >> > > > > > > > It would not be a bad idea to enhance plugin.xml
in the
> >> future
> >> > to
> >> > > > > > include
> >> > > > > > > > information about testing (i.e. Directory
containing tests
> >> > files,
> >> > > > > test
> >> > > > > > > > command, etc..)
> >> > > > > > > >
> >> > > > > > > > --Carlos
> >> > > > > > > >
> >> > > > > > > > On Thursday, September 26, 2013, Anis KADRI
wrote:
> >> > > > > > > >
> >> > > > > > > > > What's the challenge of having us use
the tests that
> come
> >> > with
> >> > > > the
> >> > > > > > > > > individual plugins ?
> >> > > > > > > > >
> >> > > > > > > > > On Thu, Sep 26, 2013 at 8:13 AM, David
Kemp <
> >> > drkemp@google.com
> >> > > > > > > > <javascript:;>>
> >> > > > > > > > > wrote:
> >> > > > > > > > > > Currently, the automated test system
that we have
> >> running
> >> > > > > (derived
> >> > > > > > > from
> >> > > > > > > > > > Medic) uses only the mobilespec
tests. It does not yet
> >> use
> >> > > > tests
> >> > > > > > > > > collected
> >> > > > > > > > > > from the plugins. Its been talked
about, but not gone
> >> > > anywhere.
> >> > > > > > > > > >
> >> > > > > > > > > > David Kemp
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > On Wed, Sep 25, 2013 at 7:58 PM,
Jesse <
> >> > > > purplecabbage@gmail.com
> >> > > > > > > > <javascript:;>>
> >> > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > >> Yeah, I have pushed some changes
to mobile-spec, and
> >> when
> >> > I
> >> > > > did
> >> > > > > I
> >> > > > > > > also
> >> > > > > > > > > >> copied the tests into the plugin
involved.
> >> > > > > > > > > >> Until we get the magic test
runner happening, I think
> >> we
> >> > > just
> >> > > > > keep
> >> > > > > > > > > >> duplicating.
> >> > > > > > > > > >>
> >> > > > > > > > > >> @purplecabbage
> >> > > > > > > > > >> risingj.com
> >> > > > > > > > > >>
> >> > > > > > > > > >>
> >> > > > > > > > > >> On Wed, Sep 25, 2013 at 4:38
PM, Steven Gill <
> >> > > > > > > stevengill97@gmail.com
> >> > > > > > > > <javascript:;>
> >> > > > > > > > > >
> >> > > > > > > > > >> wrote:
> >> > > > > > > > > >>
> >> > > > > > > > > >> > We copied over tests into
plugins when we first
> broke
> >> > them
> >> > > > > out,
> >> > > > > > > but
> >> > > > > > > > I
> >> > > > > > > > > >> don't
> >> > > > > > > > > >> > believe they have been
updated.
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > I would say for now to
just add the tests to mobile
> >> > spec,
> >> > > > and
> >> > > > > > > > > possibly in
> >> > > > > > > > > >> > the future we go all voltron
to build mobile spec
> and
> >> > keep
> >> > > > > tests
> >> > > > > > > > with
> >> > > > > > > > > >> their
> >> > > > > > > > > >> > corresponding plugins.
> >> > > > > > > > > >> >
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > On Wed, Sep 25, 2013 at
4:22 PM, Joe Bowser <
> >> > > > > bowserj@gmail.com
> >> > > > > > > > <javascript:;>>
> >> > > > > > > > > wrote:
> >> > > > > > > > > >> >
> >> > > > > > > > > >> > > Hey
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > Right now, I'm working
on a weird file issue that
> >> > > requires
> >> > > > > me
> >> > > > > > to
> >> > > > > > > > > >> > > update mobile-spec,
but I'm wondering where the
> >> tests
> >> > > > should
> >> > > > > > > live.
> >> > > > > > > > > >> > > Should it all keep
living in mobile-spec, or is
> it
> >> > with
> >> > > > the
> >> > > > > > > > plugins.
> >> > > > > > > > > >> > > And if it's with
the plugins, will there be
> >> scripts to
> >> > > > > > assemble
> >> > > > > > > > > >> > > mobile-spec all Voltron
style?
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > This came up earlier,
but I haven't found any fix
> >> that
> >> > > > > needed
> >> > > > > > a
> >> > > > > > > > > >> > > mobile-spec test.
 (Many that need native
> testing,
> >> > like
> >> > > > > > > recursive
> >> > > > > > > > > file
> >> > > > > > > > > >> > > copy, etc).  Any
thoughts?
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> > > Joe
> >> > > > > > > > > >> > >
> >> > > > > > > > > >> >
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > --
> >> > > > > > > > Carlos Santana
> >> > > > > > > > <csantana23@gmail.com>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message