cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Kemp <drk...@chromium.org>
Subject Re: mobile-spec and releases: How do we test?
Date Tue, 15 Oct 2013 16:50:46 GMT
In spite of that fact that it needs a tooling change, I like the added
<mode> tag / prepare steps.
The tooling change should be small and it means no runtime impact on apps.

I love the approach - a very positive step to cleaning up testing.

David Kemp



On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny <mmocny@chromium.org> wrote:

> Braden, you're right, good catch.
>
> Discussing locally how we could support "prepare --mode=..." in the most
> generalized form, we remembered an old suggestion to just support <mode>
> tags.
>
> The benefits seem to be:
> - No need to add custom tag-prefix/attributes for the combinations of
> js-module mode=, asset mode=, etc etc
> - More visually apparent when reading a plugin.xml file, akin to
> <platforms> tag
>
> The drawbacks seem to be:
> - Not all descendant tags are easy to support for a given mode (ie,
> <dependency>)
>
>
> Summarizing the options currently discussed in this thread:
>
> - new <js-test-module> tag.  Not general enough solution to support tests
> bundling <assets>, so -1 from me for this reason.
> - mode="..." attribute for a set of whitelisted tags (<js-module>, <asset>,
> ...)
> - <mode name="..."> tag for a set of whitelisted descendant
> tags (<js-module>, <asset>, ...)
>
> The last two options are really functionally equivalent, but given that we
> opted for <platform> tag instead of attribute, I'm also favoring a <mode>
> tag.
>
> -Michal
>
>
> On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson <braden@chromium.org
> >wrote:
>
> > It's not true that adding these tests only creates larger binaries. They
> > will be fetched and parsed by the plugin loader code at app startup time.
> > It goes through the list of all plugins in cordova_plugins.js and loads
> > them all. That parses them, and runs the outermost layer, which is the
> > wrapping function function(require, module, exports) { ... }. So there is
> > still runtime memory and CPU impact.
> >
> > I support <js-test-module> tags or <js-module ... test> or whatever.
> > Similarly, prepare for tests. I realize we want to avoid tooling support,
> > but I think tooling support is a lesser evil than production performance
> > impact.
> >
> > Overall approach makes me happy.
> >
> > Braden
> >
> >
> > On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny <mmocny@chromium.org>
> wrote:
> >
> >> On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve <agrieve@chromium.org>
> >> wrote:
> >>
> >> > The eval of the jasmine interface deserves mention. Is the motivation
> >> > there that tests can choose to use another testing framework? That's
> why
> >> > you don't just make jasmine functions globals?
> >> >
> >>
> >> I was hoping the plugins would need to depend on anything but CDVTest,
> and
> >> not expect any globals.  I guess, though, that CDVTest still expects the
> >> app to provide to a test framework and some other stuff, so in the end
> its
> >> no different.  I was hedging on being able to update CDVTest in the
> future
> >> for whatever we need, and all 3rdparty plugins would not need updating.
> >>  eval() could be used to do all sorts of clever things if we needed..
> >>
> >>
> >> >
> >> > One nit pick just from reading your email is that this will cause the
> >> test
> >> > js-modules to be injected into apps that use the plugins. I think
> >> probably
> >> > we will want to update the tools to recognize a <js-test-module>.
We
> >> > *could* work around it by adding the tests to new plugins that depend
> on
> >> > the thing they are testing, but I think changing the tools would be
> >> nicer.
> >> >
> >>
> >> I also mentioned splitting tests into second plugin but thats overkill
> >> except in extreme circumstances.  Note that the tests aren't actually
> >> loaded unless you require them, so its just a matter of larger binaries
> >> which could be filtered out manually.
> >>
> >> My person preference would be to support conditional build types, which
> >> have come up before.  ie, cordova prepare debug, cordova prepare
> release,
> >> cordova prepare test -- and plugin.xml could specify a different set of
> >> js-module for either.
> >>
> >> A specific <js-test-module> would be fine, but isnt "0 new tooling".
> >>
> >> Also, I forgot to mention, but we do need to add support for getting the
> >> full list of plugins installed, which should be trivial to add to
> >> modulemapper (maybe there  is already a way to reach in, but I don't
> think
> >> we have a documented api for it).
> >>
> >>
> >> > Another nit is that it would be nice if the CordovaTests app didn't
> >> depend
> >> > on any plugins. e.g., don't have it depend on device plugin.
> >> >
> >>
> >> CordovaTests doesn't explicitly depend on device plugin, except that at
> >> the
> >> moment I have it printing the same info that MobileSpec does at startup.
> >>  This could be wrapped in a try{}catch in case the require fails, but
> its
> >> good info to have in the common case.
> >>
> >>
> >> >
> >> > Overall, really like the approach!
> >> >
> >>
> >> Thanks!
> >>
> >>
> >> >
> >> >
> >> > On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny <mmocny@chromium.org>
> >> wrote:
> >> >
> >> >> TLDR; I've implemented a plugin testing strategy that requires 0 new
> >> >> tooling features, can support non-core plugins, and hopefully even
> >> support
> >> >> a variety of methods for running/reporting the test results.  Super
> >> alpha
> >> >> preview, but take a look if you like the direction!
> >> >>
> >> >>
> >> >> NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
> >> >>
> >> >> NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
> >> >>
> >> >> UPDATED: Converted three existing plugins to this "new style".  Code
> >> is on
> >> >> feature branches of respective repos (cdvtest branch):
> >> >> org.apache.cordova.device
> >> >> org.apache.cordova.device-motion
> >> >> org.chromium.storage
> >> >> (must clone locally, switch to branch, and plugin add from local
> path)
> >> >>
> >> >>
> >> >> Now, any plugin that wants to join in on the fun needs to provide a
> >> >> <js-module
> >> >> src="..." name="test" /> and use this template:
> >> >>
> >> >> exports.init = function() {
> >> >>
> >> >>
> >>
> eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
> >> >> 'this'));
> >> >>
> >> >>   // TESTS GO HERE
> >> >>   describe(..., function() {
> >> >>     it(...);
> >> >>   });
> >> >> };
> >> >> (The eval is optional but super useful; it will inject the jasmine
> >> >> interface into your scope, so you don't have to type
> jasmine.describe,
> >> >> jasmine.it, etc.  Not sure of any cleaner way to do this.)
> >> >>
> >> >>
> >> >> STEPS:
> >> >> 1. create a new cordova project
> >> >> 2. import the CordovaTest app into your www
> >> >> 3. add any or all of the above plugins
> >> >> 4. give it a run, and try out the auto tests (all pass on ipad, some
> >> still
> >> >> fail on android)
> >> >>
> >> >> Lots of work left to do, but hopefully good enough to whet your
> >> appetite.
> >> >>
> >> >> One thing to note, I've attempted to make CDVTest and plugin tests
> >> unaware
> >> >> of the specific jasmine version the app is hosting (so that it can
be
> >> >> swapped without changing all plugins), but it must support the new
> >> style
> >> >> interface for async tests (ie, accept a done callback).  This is the
> >> style
> >> >> that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use
> >> (I'm
> >> >> using jasmine 2.0 rc3).  This means that core plugin tests require
> some
> >> >> code transformations, but the net effect is cleaner tests and more
> >> common
> >> >> style with our node tools' tests.
> >> >>
> >> >> Manual tests are not implemented yet.
> >> >>
> >> >> -Michal
> >> >>
> >> >> On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny <mmocny@chromium.org>
> >> >> wrote:
> >> >>
> >> >> > I'm throwing something together right now, actually.  I'll post
my
> >> >> current
> >> >> > progress today so you can take a look.
> >> >> >
> >> >> >
> >> >> > On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux <b@brian.io>
wrote:
> >> >> >
> >> >> >> Sorry keep meaning to respond. I like Michal's first step
but
> >> growing
> >> >> to a
> >> >> >> full suite of tools. Are you currently tackling this Braden?
I
> feel
> >> >> like
> >> >> >> it
> >> >> >> is related to the Medic stuff and maybe we should throw one
of our
> >> >> guys on
> >> >> >> the problem fully.
> >> >> >>
> >> >> >>
> >> >> >> On Sep 27, 2013 5:10 PM, "Braden Shepherdson" <
> braden@chromium.org>
> >> >> >> wrote:
> >> >> >>
> >> >> >> > Which one?
> >> >> >> >
> >> >> >> >
> >> >> >> > On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux <b@brian.io>
> >> wrote:
> >> >> >> >
> >> >> >> > > I really like your proposal as a starting point.
Very simple
> but
> >> >> would
> >> >> >> > > allow for in-app testing as well as on the cmd line
if we so
> >> wish.
> >> >> >> > >
> >> >> >> > >
> >> >> >> > > On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny <
> >> mmocny@chromium.org
> >> >> >
> >> >> >> > wrote:
> >> >> >> > >
> >> >> >> > > > I was looking over some old emails from this
list on plugin
> >> >> testing,
> >> >> >> > and
> >> >> >> > > an
> >> >> >> > > > idea that was proposed way back was to ship
plugin tests as
> a
> >> >> second
> >> >> >> > > > plugin.  That way, you can chose to install
tests, or not,
> and
> >> >> know
> >> >> >> > > > explicitly if they are being copied into your
final project.
> >> >> >> > > >
> >> >> >> > > > An alternative would be to support build targets
a la
> >> >> >> "release/debug"
> >> >> >> > and
> >> >> >> > > > have target-specific plugin.xml tags (assets,
js-modules,
> >> >> >> > source-file..).
> >> >> >> > > >
> >> >> >> > > > -Michal
> >> >> >> > > >
> >> >> >> > > >
> >> >> >> > > > On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux
<b@brian.io>
> >> >> wrote:
> >> >> >> > > >
> >> >> >> > > > > I think this is basically what we've been
proposing for a
> >> while
> >> >> >> now.
> >> >> >> > > > >
> >> >> >> > > > >
> >> >> >> > > > > On Thu, Sep 26, 2013 at 8:29 PM, Michal
Mocny <
> >> >> >> mmocny@chromium.org>
> >> >> >> > > > wrote:
> >> >> >> > > > >
> >> >> >> > > > > > I would suggest perhaps a simpler
approach, which
> doesn't
> >> add
> >> >> >> > > anything
> >> >> >> > > > > new
> >> >> >> > > > > > to cordova-cli/plugman:
> >> >> >> > > > > >
> >> >> >> > > > > > - Each plugin ships with a "tests"
js-module, and we
> >> >> document a
> >> >> >> > > > > convention
> >> >> >> > > > > > of where they should live, and what
signature it should
> >> have
> >> >> >> (i.e.,
> >> >> >> > > > > > cordova.require('plugin.name.Tests').forEach(...)
).
> >> >> >> > > > > >   - Will need a common way to describe/report
results
> >> (others
> >> >> >> have
> >> >> >> > > > > > mentioned TAP).
> >> >> >> > > > > > - Any app is free to run those plugin
tests in any which
> >> way,
> >> >> >> but
> >> >> >> > we
> >> >> >> > > > > ship a
> >> >> >> > > > > > mobile-spec app which is one opinionated
way to do so.
> >> >> >> > > > > >   - It attempts to require the test
module for each
> >> installed
> >> >> >> > plugin,
> >> >> >> > > > > runs
> >> >> >> > > > > > them, and aggregates results.
> >> >> >> > > > > >   - It could report results to some
shared server, allow
> >> >> >> toggling
> >> >> >> > of
> >> >> >> > > > > tests,
> >> >> >> > > > > > etc, but no plugin should know or
care about those
> >> features.
> >> >> >> > > > > >
> >> >> >> > > > > > Using that as a generic base:
> >> >> >> > > > > >
> >> >> >> > > > > > - We ship a "CDVTests" (or whatever)
plugin which has a
> >> >> bunch of
> >> >> >> > > > library
> >> >> >> > > > > > code for creating tests, and plugins
can use it to
> >> register
> >> >> >> their
> >> >> >> > > > tests.
> >> >> >> > > > > > - This makes it easier to register
manual tests in a
> >> common
> >> >> >> format
> >> >> >> > > for
> >> >> >> > > > > core
> >> >> >> > > > > > plugins, and prevents code duplication
for core auto
> >> tests.
> >> >> >> > > > > > - External plugins can chose to use
our testing library,
> >> or
> >> >> not.
> >> >> >> > > > > >
> >> >> >> > > > > > -Michal
> >> >> >> > > > > >
> >> >> >> > > > > >
> >> >> >> > > > > > On Thu, Sep 26, 2013 at 10:34 AM,
Braden Shepherdson <
> >> >> >> > > > > braden@chromium.org
> >> >> >> > > > > > >wrote:
> >> >> >> > > > > >
> >> >> >> > > > > > > Here's an off-the-top-of-my-head
sketch of how we
> might
> >> do
> >> >> >> > Voltron
> >> >> >> > > > > tests:
> >> >> >> > > > > > >
> >> >> >> > > > > > > - Add a tag to plugin.xml that
names each test file:
> >> >> >> > > > > > >     <test type="automatic"
src="spec/foo.js" name="Foo
> >> >> >> Automated"
> >> >> >> > > />
> >> >> >> > > > > > >     <test type="manual" src="spec/bar.js"
name="Foo
> >> >> Manual" />
> >> >> >> > > > > > > - Add a new command, cordova
test (maybe
> prepare-test),
> >> >> that:
> >> >> >> > > > > > >     - Ignores the top-level
www.
> >> >> >> > > > > > >     - Instead copies in a basic
testing index.html
> >> similar
> >> >> to
> >> >> >> the
> >> >> >> > > > > current
> >> >> >> > > > > > > mobile-spec's
> >> >> >> > > > > > >     - That index reads a file
akin to
> cordova_plugins.js
> >> >> >> > > > > > (cordova_tests.js,
> >> >> >> > > > > > > maybe?) generated by the CLI,
containing the info from
> >> the
> >> >> >> <test>
> >> >> >> > > > tags.
> >> >> >> > > > > > >     - It has navigation similar
to the current
> >> mobile-spec,
> >> >> >> with
> >> >> >> > > > > buttons
> >> >> >> > > > > > > for the automatic and manual
sections. Auto has "All"
> >> and
> >> >> then
> >> >> >> > each
> >> >> >> > > > > > module,
> >> >> >> > > > > > > manual just has the list of
modules.
> >> >> >> > > > > > >
> >> >> >> > > > > > > Thoughts?
> >> >> >> > > > > > >
> >> >> >> > > > > > > Braden
> >> >> >> > > > > > >
> >> >> >> > > > > > >
> >> >> >> > > > > > > On Thu, Sep 26, 2013 at 6:33
AM, Carlos Santana <
> >> >> >> > > > csantana23@gmail.com
> >> >> >> > > > > > > >wrote:
> >> >> >> > > > > > >
> >> >> >> > > > > > > > I like the idea can we
call mobilespec now
> >> >> cordova-voltron
> >> >> >> and
> >> >> >> > be
> >> >> >> > > > DRY
> >> >> >> > > > > > and
> >> >> >> > > > > > > > use the tests form the
plugins.
> >> >> >> > > > > > > >
> >> >> >> > > > > > > > Voltron by itself creates
an App that tests only
> core,
> >> >> but
> >> >> >> as
> >> >> >> > you
> >> >> >> > > > > > > > use plugman to add plugins
to voltron it has more
> test
> >> >> >> cases.
> >> >> >> > > > > > > >
> >> >> >> > > > > > > > It would not be a bad idea
to enhance plugin.xml in
> >> the
> >> >> >> future
> >> >> >> > to
> >> >> >> > > > > > include
> >> >> >> > > > > > > > information about testing
(i.e. Directory containing
> >> >> tests
> >> >> >> > files,
> >> >> >> > > > > test
> >> >> >> > > > > > > > command, etc..)
> >> >> >> > > > > > > >
> >> >> >> > > > > > > > --Carlos
> >> >> >> > > > > > > >
> >> >> >> > > > > > > > On Thursday, September
26, 2013, Anis KADRI wrote:
> >> >> >> > > > > > > >
> >> >> >> > > > > > > > > What's the challenge
of having us use the tests
> that
> >> >> come
> >> >> >> > with
> >> >> >> > > > the
> >> >> >> > > > > > > > > individual plugins
?
> >> >> >> > > > > > > > >
> >> >> >> > > > > > > > > On Thu, Sep 26, 2013
at 8:13 AM, David Kemp <
> >> >> >> > drkemp@google.com
> >> >> >> > > > > > > > <javascript:;>>
> >> >> >> > > > > > > > > wrote:
> >> >> >> > > > > > > > > > Currently, the
automated test system that we
> have
> >> >> >> running
> >> >> >> > > > > (derived
> >> >> >> > > > > > > from
> >> >> >> > > > > > > > > > Medic) uses only
the mobilespec tests. It does
> not
> >> >> yet
> >> >> >> use
> >> >> >> > > > tests
> >> >> >> > > > > > > > > collected
> >> >> >> > > > > > > > > > from the plugins.
Its been talked about, but not
> >> gone
> >> >> >> > > anywhere.
> >> >> >> > > > > > > > > >
> >> >> >> > > > > > > > > > David Kemp
> >> >> >> > > > > > > > > >
> >> >> >> > > > > > > > > >
> >> >> >> > > > > > > > > > On Wed, Sep 25,
2013 at 7:58 PM, Jesse <
> >> >> >> > > > purplecabbage@gmail.com
> >> >> >> > > > > > > > <javascript:;>>
> >> >> >> > > > > > > > > wrote:
> >> >> >> > > > > > > > > >
> >> >> >> > > > > > > > > >> Yeah, I have
pushed some changes to
> mobile-spec,
> >> and
> >> >> >> when
> >> >> >> > I
> >> >> >> > > > did
> >> >> >> > > > > I
> >> >> >> > > > > > > also
> >> >> >> > > > > > > > > >> copied the
tests into the plugin involved.
> >> >> >> > > > > > > > > >> Until we
get the magic test runner happening, I
> >> >> think
> >> >> >> we
> >> >> >> > > just
> >> >> >> > > > > keep
> >> >> >> > > > > > > > > >> duplicating.
> >> >> >> > > > > > > > > >>
> >> >> >> > > > > > > > > >> @purplecabbage
> >> >> >> > > > > > > > > >> risingj.com
> >> >> >> > > > > > > > > >>
> >> >> >> > > > > > > > > >>
> >> >> >> > > > > > > > > >> On Wed, Sep
25, 2013 at 4:38 PM, Steven Gill <
> >> >> >> > > > > > > stevengill97@gmail.com
> >> >> >> > > > > > > > <javascript:;>
> >> >> >> > > > > > > > > >
> >> >> >> > > > > > > > > >> wrote:
> >> >> >> > > > > > > > > >>
> >> >> >> > > > > > > > > >> > We copied
over tests into plugins when we
> first
> >> >> broke
> >> >> >> > them
> >> >> >> > > > > out,
> >> >> >> > > > > > > but
> >> >> >> > > > > > > > I
> >> >> >> > > > > > > > > >> don't
> >> >> >> > > > > > > > > >> > believe
they have been updated.
> >> >> >> > > > > > > > > >> >
> >> >> >> > > > > > > > > >> > I would
say for now to just add the tests to
> >> >> mobile
> >> >> >> > spec,
> >> >> >> > > > and
> >> >> >> > > > > > > > > possibly in
> >> >> >> > > > > > > > > >> > the
future we go all voltron to build mobile
> >> spec
> >> >> and
> >> >> >> > keep
> >> >> >> > > > > tests
> >> >> >> > > > > > > > with
> >> >> >> > > > > > > > > >> their
> >> >> >> > > > > > > > > >> > corresponding
plugins.
> >> >> >> > > > > > > > > >> >
> >> >> >> > > > > > > > > >> >
> >> >> >> > > > > > > > > >> > On Wed,
Sep 25, 2013 at 4:22 PM, Joe Bowser <
> >> >> >> > > > > bowserj@gmail.com
> >> >> >> > > > > > > > <javascript:;>>
> >> >> >> > > > > > > > > wrote:
> >> >> >> > > > > > > > > >> >
> >> >> >> > > > > > > > > >> > >
Hey
> >> >> >> > > > > > > > > >> > >
> >> >> >> > > > > > > > > >> > >
Right now, I'm working on a weird file
> issue
> >> >> that
> >> >> >> > > requires
> >> >> >> > > > > me
> >> >> >> > > > > > to
> >> >> >> > > > > > > > > >> > >
update mobile-spec, but I'm wondering where
> >> the
> >> >> >> tests
> >> >> >> > > > should
> >> >> >> > > > > > > live.
> >> >> >> > > > > > > > > >> > >
Should it all keep living in mobile-spec,
> or
> >> is
> >> >> it
> >> >> >> > with
> >> >> >> > > > the
> >> >> >> > > > > > > > plugins.
> >> >> >> > > > > > > > > >> > >
And if it's with the plugins, will there be
> >> >> >> scripts to
> >> >> >> > > > > > assemble
> >> >> >> > > > > > > > > >> > >
mobile-spec all Voltron style?
> >> >> >> > > > > > > > > >> > >
> >> >> >> > > > > > > > > >> > >
This came up earlier, but I haven't found
> any
> >> >> fix
> >> >> >> that
> >> >> >> > > > > needed
> >> >> >> > > > > > a
> >> >> >> > > > > > > > > >> > >
mobile-spec test.  (Many that need native
> >> >> testing,
> >> >> >> > like
> >> >> >> > > > > > > recursive
> >> >> >> > > > > > > > > file
> >> >> >> > > > > > > > > >> > >
copy, etc).  Any thoughts?
> >> >> >> > > > > > > > > >> > >
> >> >> >> > > > > > > > > >> > >
Joe
> >> >> >> > > > > > > > > >> > >
> >> >> >> > > > > > > > > >> >
> >> >> >> > > > > > > > > >>
> >> >> >> > > > > > > > >
> >> >> >> > > > > > > >
> >> >> >> > > > > > > >
> >> >> >> > > > > > > > --
> >> >> >> > > > > > > > Carlos Santana
> >> >> >> > > > > > > > <csantana23@gmail.com>
> >> >> >> > > > > > > >
> >> >> >> > > > > > >
> >> >> >> > > > > >
> >> >> >> > > > >
> >> >> >> > > >
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >> >
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message