cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse <purplecabb...@gmail.com>
Subject Re: plugin test framework
Date Thu, 19 Jun 2014 20:53:39 GMT
re:

   Q: What do I do if my plugin tests must have very large assets?
   - A: Don't bundle those assets with your plugin. If you can, have your
...


My concern is I do not want to see tests added to every project that uses a
plugin, even if the assets are not large, there are implications to
including the test framework + all the tests because they get loaded and
processed with all of the plugins and will impact load time even if never
run.

99.9% of the time the plugin tests will be used by us the plugin
developers, and not the people who use the plugin in there apps.

I agree, having the tester install the test harness plugin dependency
themselves is probably a better option, as I see you have wrapped all tests
inside a exports.defineAutoTests so we don't have to worry about
describe/it/expects not being defined.


@purplecabbage
risingj.com


On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmocny@chromium.org> wrote:

> Jesee, the branch is NOT a requirement (I don't think I documented it as
> such, except in the examples for installing plugins for initial look).
>  Actually, we should delete those stale branches now that we are moving
> up-to-date tests into master.  It was just for early experimentation on the
> feature.
>
> Jesse, I'm not seeing the benefit of using plugins-tests.xml or the
> dependency on the test plugin yet, may you elaborate your thoughts?
>
> My hope was that tests are just always installed alongside plugins.  If
> that is not a good idea for some particular plugin, say because it uses
> huge assets, I elaborated my answer in the plugin FAQ (
>
> https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq
> ):
>  FAQ
>
>    -
>
>    Q: Should I add org.apache.cordova.test-harness as a <dependancy> of my
>    plugin?
>    - A: No. The end-user should decide if they want to install the test
>       harness, not your plugin (most users won't).
>    -
>
>    Q: What do I do if my plugin tests must have very large assets?
>    - A: Don't bundle those assets with your plugin. If you can, have your
>       tests fail gracefully if those assets don't don't exist (perhaps log
> a
>       warning, perhaps fail a single asset-checking test, and skip the
> rest).
>       Then, ideally download those assets automatically into local storage
> the
>       first time tests run. Or create a manual test step to download
> and install
>       assets. As a final alternative, split those test assets into a
> separate
>       plugin, and instruct users to install that plugin to run your full
> test
>       suite.
>    -
>
>    Q: Should I ship my app with the test harness plugin installed?
>    - A: Not likely. If you want, you can. Then your app could even embed a
>       link to the test page (cdvtests/index.html) from a help section of
>       your app, to give end users a way to run your test suite out in
> the feild.
>       That may help diagnose causes of issues within your app. Maybe.
>
>
> =============
>
>
> Feel free the debate those answers -- now is certainly the time -- but I
> put a lot of effort to make it super flexible and to not require depending
> on changes to CLI :P
>
> -Michal
>
>
> On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabbage@gmail.com> wrote:
>
> > Sorry I missed providing feedback on this earlier ...
> > Having a deeper look at this, I am not feeling great about the extra
> > requirement that every plugin have an additional branch.
> >
> > Several concerns arise :
> > - test branch can be out of sync with master
> > - how do we test a specific version?
> > - tests are not immediately visible when looking at master
> > - differing versions of plugin.xml depending on the branch
> >
> > The majority of the work has been done (thanks Michal!), and mostly any
> > suggestions I make will just require moving code and changing a few
> > conventions.
> >
> > What if we? :
> > 1. add a plugin-test.xml file which has the exact format of plugin.xml
> > 2. keep tests/ and plugin-test.xml file in master branch
> > 3. have plugman/cli support an additional flag --test so we can install
> > like this:
> > cordova plugin add
> > http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git --test
> > This would mean that in addition to processing the plugin.xml of the
> > plugin, we would also process the plugin-test.xml file ( identical
> > processing logic )
> > 4. have all plugin-test.xml files declare a dependency on
> > cordova-plugin-test-framework
> >
> > The above suggestions could also be used in conjuction with the cordova
> run
> > --tests platform mentioned by Michal, but without the need to manage the
> > switching of branches.
> >
> >
> > @purplecabbage
> > risingj.com
> >
> >
> > On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny <mmocny@chromium.org>
> wrote:
> >
> > > Piotr: Actually I'm not sure how running tests in the harness would
> work,
> > > since the path to the resource may be different.  However, in general,
> > with
> > > development using the harness you aren't making any changes to plugins.
> > >  The whole point is for app developers who want to modify only web
> > > application bits and not deal with native compiles.
> > >
> > > In theory the app harness could support working on the js-modules of
> > > plugins, but that sounds like a really niche idea.  I'd not be opposed
> to
> > > someone working on it but I'm not sure you'll have luck finding
> > volunteers.
> > >
> > > -Michal
> > >
> > >
> > > On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny <mmocny@chromium.org>
> > wrote:
> > >
> > > > At the time I went through my design iterations I just didn't want to
> > > > necessarily depend on cordova tooling changes / documentation.  In
> > other
> > > > words, someone else may have a different strategy for testing..
> > > >
> > > > My personal opinion would be have the test plugin ship with a plugin
> > hook
> > > > (those are in, right? or at least on their way), that will
> > automatically
> > > > update the start page if you pass a flag to run command.  Ie, in an
> > ideal
> > > > world:  `cordova run --tests` runs a plugin hook passing in --tests
> > flag
> > > > which changes the start page, in a way that isn't overwritten by the
> > > > top-level config.xml.
> > > >
> > > > My 2 cents, since I don't want "our way" of testing mobile spec to be
> > > "the
> > > > only way" to test.   Frameworks and opinions on testing change.
> > > >
> > > > -Michal
> > > >
> > > >
> > > > On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa <pzalewa@mozilla.com>
> > > wrote:
> > > >
> > > >> One thing more - it would be great if user could create a test using
> > > test
> > > >> harness app as well. Is it also considered?
> > > >>
> > > >> Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez pisze:
> > > >>
> > > >>  It would be a nice to have in the cli, aimed to just setup the
> right
> > > path
> > > >>> in the config.xml, maybe along with an another argument to build,
> > > >>> run/emulate as well.
> > > >>> It sounds great.
> > > >>>
> > > >>>
> > > >>> 2014-06-17 15:21 GMT-05:00 Piotr Zalewa <pzalewa@mozilla.com>:
> > > >>>
> > > >>>  Thanks Martin,
> > > >>>>
> > > >>>> Has it been considered to create a separate command "testrun"
or
> > > similar
> > > >>>> which would remove the need to edit the config.xml?
> > > >>>>
> > > >>>> Dnia Tue Jun 17 11:58:33 2014 Michal Mocny pisze:
> > > >>>>
> > > >>>>   Martin, thanks for posting those links.
> > > >>>>
> > > >>>>>
> > > >>>>> And I'll look into the INFRA tickets I need to file to
set up a
> > repo
> > > >>>>> for
> > > >>>>> that plugin, since its ready to come out of labs.
> > > >>>>>
> > > >>>>>
> > > >>>>> On Tue, Jun 17, 2014 at 2:06 PM, Martin Gonzalez <
> > > >>>>> martin.c.glez.glez@gmail.com> wrote:
> > > >>>>>
> > > >>>>>   This is the Cordova Plugin Test Framework readme.md,
you can
> > catch
> > > >>>>> up
> > > >>>>>
> > > >>>>>> with
> > > >>>>>> the functionality by reading some of the content:
> > > >>>>>>
> > > >>>>>> Repository:
> > > >>>>>> https://github.com/apache/cordova-labs
> > > >>>>>>
> > > >>>>>> Docs:
> > > >>>>>> https://github.com/apache/cordova-labs/blob/master/README.md
> > > >>>>>>
> > > >>>>>> https://github.com/apache/cordova-labs/blob/cdvtest/
> > > >>>>>> cordova-plugin-test-framework/README.md
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 2014-06-17 12:56 GMT-05:00 Piotr Zalewa <pzalewa@mozilla.com>:
> > > >>>>>>
> > > >>>>>>   a documentation explaining how it's gonna work
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> Dnia Tue Jun 17 10:51:58 2014 Michal Mocny pisze:
> > > >>>>>>>
> > > >>>>>>>    What do you mean?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Tue, Jun 17, 2014 at 1:50 PM, Piotr Zalewa
<
> > > pzalewa@mozilla.com>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>    Is there any predev document?
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> Dnia Mon Jun 16 18:30:46 2014 Andrew Grieve
pisze:
> > > >>>>>>>>>
> > > >>>>>>>>>     Yeah, really exciting. Thanks for
taking this on.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>> On Mon, Jun 16, 2014 at 3:42 PM, Michal
Mocny <
> > > >>>>>>>>>> mmocny@chromium.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>     Fantastic!
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>  I'll try to keep an eye out on the
PR's, and please ping me
> > if
> > > >>>>>>>>>>> you
> > > >>>>>>>>>>> would
> > > >>>>>>>>>>> like any help.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> -Michal
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Mon, Jun 16, 2014 at 3:25 PM,
Marcel Kinard <
> > > >>>>>>>>>>> cmarcelk@gmail.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>     Hi, after some discussions
here with IBM management,
> > we’re
> > > >>>>>>>>>>> going
> > > >>>>>>>>>>> to
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>   bring
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> in a couple extra interns
for a week to jumpstart the
> > > migration
> > > >>>>>>>>>>>> of
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>  the
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>  tests out of mobile-spec into the new plugin test
framework.
> > Staci
> > > >>>>>>>
> > > >>>>>>>>  Cooper
> > > >>>>>>>>>>>> will be leading this effort,
and Martin Gonzalez will be a
> > > part
> > > >>>>>>>>>>>> of
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>  it.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>
> > > >>>>>>>  So if you see a bunch of pull requests, this
is what it is
> for.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>  We’ll
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>  get
> > > >>>>>>>
> > > >>>>>>>>  the interns to submit an ICLA asap.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>     --
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>  Piotr Zalewa
> > > >>>>>>>>> Mozilla
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>    --
> > > >>>>>>>>
> > > >>>>>>> Piotr Zalewa
> > > >>>>>>> Mozilla
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> Regards,
> > > >>>>>> Martin Gonzalez
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>  --
> > > >>>> Piotr Zalewa
> > > >>>> Mozilla
> > > >>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >>>
> > > >> --
> > > >> Piotr Zalewa
> > > >> Mozilla
> > > >>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message