cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Braden Shepherdson <bra...@chromium.org>
Subject Re: mobile-spec and releases: How do we test?
Date Tue, 15 Oct 2013 15:43:48 GMT
It's not true that adding these tests only creates larger binaries. They
will be fetched and parsed by the plugin loader code at app startup time.
It goes through the list of all plugins in cordova_plugins.js and loads
them all. That parses them, and runs the outermost layer, which is the
wrapping function function(require, module, exports) { ... }. So there is
still runtime memory and CPU impact.

I support <js-test-module> tags or <js-module ... test> or whatever.
Similarly, prepare for tests. I realize we want to avoid tooling support,
but I think tooling support is a lesser evil than production performance
impact.

Overall approach makes me happy.

Braden


On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny <mmocny@chromium.org> wrote:

> On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve <agrieve@chromium.org>
> wrote:
>
> > The eval of the jasmine interface deserves mention. Is the motivation
> > there that tests can choose to use another testing framework? That's why
> > you don't just make jasmine functions globals?
> >
>
> I was hoping the plugins would need to depend on anything but CDVTest, and
> not expect any globals.  I guess, though, that CDVTest still expects the
> app to provide to a test framework and some other stuff, so in the end its
> no different.  I was hedging on being able to update CDVTest in the future
> for whatever we need, and all 3rdparty plugins would not need updating.
>  eval() could be used to do all sorts of clever things if we needed..
>
>
> >
> > One nit pick just from reading your email is that this will cause the
> test
> > js-modules to be injected into apps that use the plugins. I think
> probably
> > we will want to update the tools to recognize a <js-test-module>. We
> > *could* work around it by adding the tests to new plugins that depend on
> > the thing they are testing, but I think changing the tools would be
> nicer.
> >
>
> I also mentioned splitting tests into second plugin but thats overkill
> except in extreme circumstances.  Note that the tests aren't actually
> loaded unless you require them, so its just a matter of larger binaries
> which could be filtered out manually.
>
> My person preference would be to support conditional build types, which
> have come up before.  ie, cordova prepare debug, cordova prepare release,
> cordova prepare test -- and plugin.xml could specify a different set of
> js-module for either.
>
> A specific <js-test-module> would be fine, but isnt "0 new tooling".
>
> Also, I forgot to mention, but we do need to add support for getting the
> full list of plugins installed, which should be trivial to add to
> modulemapper (maybe there  is already a way to reach in, but I don't think
> we have a documented api for it).
>
>
> > Another nit is that it would be nice if the CordovaTests app didn't
> depend
> > on any plugins. e.g., don't have it depend on device plugin.
> >
>
> CordovaTests doesn't explicitly depend on device plugin, except that at the
> moment I have it printing the same info that MobileSpec does at startup.
>  This could be wrapped in a try{}catch in case the require fails, but its
> good info to have in the common case.
>
>
> >
> > Overall, really like the approach!
> >
>
> Thanks!
>
>
> >
> >
> > On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny <mmocny@chromium.org>
> wrote:
> >
> >> TLDR; I've implemented a plugin testing strategy that requires 0 new
> >> tooling features, can support non-core plugins, and hopefully even
> support
> >> a variety of methods for running/reporting the test results.  Super
> alpha
> >> preview, but take a look if you like the direction!
> >>
> >>
> >> NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
> >>
> >> NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
> >>
> >> UPDATED: Converted three existing plugins to this "new style".  Code is
> on
> >> feature branches of respective repos (cdvtest branch):
> >> org.apache.cordova.device
> >> org.apache.cordova.device-motion
> >> org.chromium.storage
> >> (must clone locally, switch to branch, and plugin add from local path)
> >>
> >>
> >> Now, any plugin that wants to join in on the fun needs to provide a
> >> <js-module
> >> src="..." name="test" /> and use this template:
> >>
> >> exports.init = function() {
> >>
> >>
> eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
> >> 'this'));
> >>
> >>   // TESTS GO HERE
> >>   describe(..., function() {
> >>     it(...);
> >>   });
> >> };
> >> (The eval is optional but super useful; it will inject the jasmine
> >> interface into your scope, so you don't have to type jasmine.describe,
> >> jasmine.it, etc.  Not sure of any cleaner way to do this.)
> >>
> >>
> >> STEPS:
> >> 1. create a new cordova project
> >> 2. import the CordovaTest app into your www
> >> 3. add any or all of the above plugins
> >> 4. give it a run, and try out the auto tests (all pass on ipad, some
> still
> >> fail on android)
> >>
> >> Lots of work left to do, but hopefully good enough to whet your
> appetite.
> >>
> >> One thing to note, I've attempted to make CDVTest and plugin tests
> unaware
> >> of the specific jasmine version the app is hosting (so that it can be
> >> swapped without changing all plugins), but it must support the new style
> >> interface for async tests (ie, accept a done callback).  This is the
> style
> >> that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use (I'm
> >> using jasmine 2.0 rc3).  This means that core plugin tests require some
> >> code transformations, but the net effect is cleaner tests and more
> common
> >> style with our node tools' tests.
> >>
> >> Manual tests are not implemented yet.
> >>
> >> -Michal
> >>
> >> On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny <mmocny@chromium.org>
> >> wrote:
> >>
> >> > I'm throwing something together right now, actually.  I'll post my
> >> current
> >> > progress today so you can take a look.
> >> >
> >> >
> >> > On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux <b@brian.io> wrote:
> >> >
> >> >> Sorry keep meaning to respond. I like Michal's first step but growing
> >> to a
> >> >> full suite of tools. Are you currently tackling this Braden? I feel
> >> like
> >> >> it
> >> >> is related to the Medic stuff and maybe we should throw one of our
> >> guys on
> >> >> the problem fully.
> >> >>
> >> >>
> >> >> On Sep 27, 2013 5:10 PM, "Braden Shepherdson" <braden@chromium.org>
> >> >> wrote:
> >> >>
> >> >> > Which one?
> >> >> >
> >> >> >
> >> >> > On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux <b@brian.io>
wrote:
> >> >> >
> >> >> > > I really like your proposal as a starting point. Very simple
but
> >> would
> >> >> > > allow for in-app testing as well as on the cmd line if we
so
> wish.
> >> >> > >
> >> >> > >
> >> >> > > On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny <
> mmocny@chromium.org
> >> >
> >> >> > wrote:
> >> >> > >
> >> >> > > > I was looking over some old emails from this list on
plugin
> >> testing,
> >> >> > and
> >> >> > > an
> >> >> > > > idea that was proposed way back was to ship plugin tests
as a
> >> second
> >> >> > > > plugin.  That way, you can chose to install tests, or
not, and
> >> know
> >> >> > > > explicitly if they are being copied into your final
project.
> >> >> > > >
> >> >> > > > An alternative would be to support build targets a la
> >> >> "release/debug"
> >> >> > and
> >> >> > > > have target-specific plugin.xml tags (assets, js-modules,
> >> >> > source-file..).
> >> >> > > >
> >> >> > > > -Michal
> >> >> > > >
> >> >> > > >
> >> >> > > > On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux <b@brian.io>
> >> wrote:
> >> >> > > >
> >> >> > > > > I think this is basically what we've been proposing
for a
> while
> >> >> now.
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny <
> >> >> mmocny@chromium.org>
> >> >> > > > wrote:
> >> >> > > > >
> >> >> > > > > > I would suggest perhaps a simpler approach,
which doesn't
> add
> >> >> > > anything
> >> >> > > > > new
> >> >> > > > > > to cordova-cli/plugman:
> >> >> > > > > >
> >> >> > > > > > - Each plugin ships with a "tests" js-module,
and we
> >> document a
> >> >> > > > > convention
> >> >> > > > > > of where they should live, and what signature
it should
> have
> >> >> (i.e.,
> >> >> > > > > > cordova.require('plugin.name.Tests').forEach(...)
).
> >> >> > > > > >   - Will need a common way to describe/report
results
> (others
> >> >> have
> >> >> > > > > > mentioned TAP).
> >> >> > > > > > - Any app is free to run those plugin tests
in any which
> way,
> >> >> but
> >> >> > we
> >> >> > > > > ship a
> >> >> > > > > > mobile-spec app which is one opinionated way
to do so.
> >> >> > > > > >   - It attempts to require the test module
for each
> installed
> >> >> > plugin,
> >> >> > > > > runs
> >> >> > > > > > them, and aggregates results.
> >> >> > > > > >   - It could report results to some shared
server, allow
> >> >> toggling
> >> >> > of
> >> >> > > > > tests,
> >> >> > > > > > etc, but no plugin should know or care about
those
> features.
> >> >> > > > > >
> >> >> > > > > > Using that as a generic base:
> >> >> > > > > >
> >> >> > > > > > - We ship a "CDVTests" (or whatever) plugin
which has a
> >> bunch of
> >> >> > > > library
> >> >> > > > > > code for creating tests, and plugins can use
it to register
> >> >> their
> >> >> > > > tests.
> >> >> > > > > > - This makes it easier to register manual
tests in a common
> >> >> format
> >> >> > > for
> >> >> > > > > core
> >> >> > > > > > plugins, and prevents code duplication for
core auto tests.
> >> >> > > > > > - External plugins can chose to use our testing
library, or
> >> not.
> >> >> > > > > >
> >> >> > > > > > -Michal
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson
<
> >> >> > > > > braden@chromium.org
> >> >> > > > > > >wrote:
> >> >> > > > > >
> >> >> > > > > > > Here's an off-the-top-of-my-head sketch
of how we might
> do
> >> >> > Voltron
> >> >> > > > > tests:
> >> >> > > > > > >
> >> >> > > > > > > - Add a tag to plugin.xml that names
each test file:
> >> >> > > > > > >     <test type="automatic" src="spec/foo.js"
name="Foo
> >> >> Automated"
> >> >> > > />
> >> >> > > > > > >     <test type="manual" src="spec/bar.js"
name="Foo
> >> Manual" />
> >> >> > > > > > > - Add a new command, cordova test (maybe
prepare-test),
> >> that:
> >> >> > > > > > >     - Ignores the top-level www.
> >> >> > > > > > >     - Instead copies in a basic testing
index.html
> similar
> >> to
> >> >> the
> >> >> > > > > current
> >> >> > > > > > > mobile-spec's
> >> >> > > > > > >     - That index reads a file akin to
cordova_plugins.js
> >> >> > > > > > (cordova_tests.js,
> >> >> > > > > > > maybe?) generated by the CLI, containing
the info from
> the
> >> >> <test>
> >> >> > > > tags.
> >> >> > > > > > >     - It has navigation similar to the
current
> mobile-spec,
> >> >> with
> >> >> > > > > buttons
> >> >> > > > > > > for the automatic and manual sections.
Auto has "All" and
> >> then
> >> >> > each
> >> >> > > > > > module,
> >> >> > > > > > > manual just has the list of modules.
> >> >> > > > > > >
> >> >> > > > > > > Thoughts?
> >> >> > > > > > >
> >> >> > > > > > > Braden
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > > On Thu, Sep 26, 2013 at 6:33 AM, Carlos
Santana <
> >> >> > > > csantana23@gmail.com
> >> >> > > > > > > >wrote:
> >> >> > > > > > >
> >> >> > > > > > > > I like the idea can we call mobilespec
now
> >> cordova-voltron
> >> >> and
> >> >> > be
> >> >> > > > DRY
> >> >> > > > > > and
> >> >> > > > > > > > use the tests form the plugins.
> >> >> > > > > > > >
> >> >> > > > > > > > Voltron by itself creates an App
that tests only core,
> >> but
> >> >> as
> >> >> > you
> >> >> > > > > > > > use plugman to add plugins to voltron
it has more test
> >> >> cases.
> >> >> > > > > > > >
> >> >> > > > > > > > It would not be a bad idea to enhance
plugin.xml in the
> >> >> future
> >> >> > to
> >> >> > > > > > include
> >> >> > > > > > > > information about testing (i.e.
Directory containing
> >> tests
> >> >> > files,
> >> >> > > > > test
> >> >> > > > > > > > command, etc..)
> >> >> > > > > > > >
> >> >> > > > > > > > --Carlos
> >> >> > > > > > > >
> >> >> > > > > > > > On Thursday, September 26, 2013,
Anis KADRI wrote:
> >> >> > > > > > > >
> >> >> > > > > > > > > What's the challenge of having
us use the tests that
> >> come
> >> >> > with
> >> >> > > > the
> >> >> > > > > > > > > individual plugins ?
> >> >> > > > > > > > >
> >> >> > > > > > > > > On Thu, Sep 26, 2013 at 8:13
AM, David Kemp <
> >> >> > drkemp@google.com
> >> >> > > > > > > > <javascript:;>>
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > > Currently, the automated
test system that we have
> >> >> running
> >> >> > > > > (derived
> >> >> > > > > > > from
> >> >> > > > > > > > > > Medic) uses only the mobilespec
tests. It does not
> >> yet
> >> >> use
> >> >> > > > tests
> >> >> > > > > > > > > collected
> >> >> > > > > > > > > > from the plugins. Its
been talked about, but not
> gone
> >> >> > > anywhere.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > David Kemp
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > On Wed, Sep 25, 2013 at
7:58 PM, Jesse <
> >> >> > > > purplecabbage@gmail.com
> >> >> > > > > > > > <javascript:;>>
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >> Yeah, I have pushed
some changes to mobile-spec,
> and
> >> >> when
> >> >> > I
> >> >> > > > did
> >> >> > > > > I
> >> >> > > > > > > also
> >> >> > > > > > > > > >> copied the tests into
the plugin involved.
> >> >> > > > > > > > > >> Until we get the magic
test runner happening, I
> >> think
> >> >> we
> >> >> > > just
> >> >> > > > > keep
> >> >> > > > > > > > > >> duplicating.
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> @purplecabbage
> >> >> > > > > > > > > >> risingj.com
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> On Wed, Sep 25, 2013
at 4:38 PM, Steven Gill <
> >> >> > > > > > > stevengill97@gmail.com
> >> >> > > > > > > > <javascript:;>
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >> wrote:
> >> >> > > > > > > > > >>
> >> >> > > > > > > > > >> > We copied over
tests into plugins when we first
> >> broke
> >> >> > them
> >> >> > > > > out,
> >> >> > > > > > > but
> >> >> > > > > > > > I
> >> >> > > > > > > > > >> don't
> >> >> > > > > > > > > >> > believe they
have been updated.
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > I would say for
now to just add the tests to
> >> mobile
> >> >> > spec,
> >> >> > > > and
> >> >> > > > > > > > > possibly in
> >> >> > > > > > > > > >> > the future we
go all voltron to build mobile
> spec
> >> and
> >> >> > keep
> >> >> > > > > tests
> >> >> > > > > > > > with
> >> >> > > > > > > > > >> their
> >> >> > > > > > > > > >> > corresponding
plugins.
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > On Wed, Sep 25,
2013 at 4:22 PM, Joe Bowser <
> >> >> > > > > bowserj@gmail.com
> >> >> > > > > > > > <javascript:;>>
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >> > > Hey
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > Right now,
I'm working on a weird file issue
> >> that
> >> >> > > requires
> >> >> > > > > me
> >> >> > > > > > to
> >> >> > > > > > > > > >> > > update mobile-spec,
but I'm wondering where
> the
> >> >> tests
> >> >> > > > should
> >> >> > > > > > > live.
> >> >> > > > > > > > > >> > > Should it
all keep living in mobile-spec, or
> is
> >> it
> >> >> > with
> >> >> > > > the
> >> >> > > > > > > > plugins.
> >> >> > > > > > > > > >> > > And if it's
with the plugins, will there be
> >> >> scripts to
> >> >> > > > > > assemble
> >> >> > > > > > > > > >> > > mobile-spec
all Voltron style?
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > This came
up earlier, but I haven't found any
> >> fix
> >> >> that
> >> >> > > > > needed
> >> >> > > > > > a
> >> >> > > > > > > > > >> > > mobile-spec
test.  (Many that need native
> >> testing,
> >> >> > like
> >> >> > > > > > > recursive
> >> >> > > > > > > > > file
> >> >> > > > > > > > > >> > > copy, etc).
 Any thoughts?
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> > > Joe
> >> >> > > > > > > > > >> > >
> >> >> > > > > > > > > >> >
> >> >> > > > > > > > > >>
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > > --
> >> >> > > > > > > > Carlos Santana
> >> >> > > > > > > > <csantana23@gmail.com>
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message