cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Zalewa <pzal...@mozilla.com>
Subject Re: plugin test framework
Date Fri, 20 Jun 2014 04:40:04 GMT
testing is good, no need to hide it,
it would be good though to not copy it with the rendered app

Dnia Thu Jun 19 19:11:25 2014 purplecabbage pisze:
> I think we just lead by example.
>
> Sent from my iPhone
>
>> On Jun 19, 2014, at 6:18 PM, Michal Mocny <mmocny@chromium.org> wrote:
>>
>> +1 I agree, this would be awesome.
>>
>> New question, should this merely be the "standard" we adhere to for core
>> plugins, or should we actively make it difficult for plugin devs to not
>> ship tests directly with plugins? (Not sure how we could accomplish that,
>> so I hope its just a convention that applies to our work).
>>
>> -Michal
>>
>>
>>> On Thu, Jun 19, 2014 at 7:48 PM, Jesse <purplecabbage@gmail.com> wrote:
>>>
>>> My ultimate would be this:
>>>
>>> cordova create TestFilePlugin
>>> cd TestFilePlugin
>>> cordova platform add android
>>> cordova plugin add
>>>
>>> http://git-wip-us.apache.org/repos/asf/cordova-labs.git#cdvtest:cordova-plugin-test-framework
>>> cordova plugin add ../cordova-plugin-file/
>>> cordova plugin add ../cordova-plugin-file/test/
>>> cordova run android --start cdvtests/index.html
>>>
>>> Then do this for each plugin, and for each platform
>>> Then do this for all combinations of plugins
>>> ...
>>>
>>> Note the run --start does not yet exist, but this would be awesome!
>>>
>>>
>>> @purplecabbage
>>> risingj.com
>>>
>>>
>>>> On Thu, Jun 19, 2014 at 4:15 PM, Jesse <purplecabbage@gmail.com> wrote:
>>>>
>>>> Option a) was what I suggested a ways back, and I still stand by it.
>>>> I think it provides the greatest transparency, and simplicity, yet it is
>>>> still very flexible.
>>>> I don't think it would be hard to accomplish either. This is the small
>>>> re-org I was hinting at, you've already done the hard part.
>>>>
>>>>
>>>> @purplecabbage
>>>> risingj.com
>>>>
>>>>
>>>>> On Thu, Jun 19, 2014 at 3:45 PM, Michal Mocny <mmocny@chromium.org>
>>>> wrote:
>>>>
>>>>> Andrew has raised that concern as well.  My gut says that the bundling
>>> of
>>>>> a
>>>>> few shorts scripts that get parsed but not run as long as they don't
get
>>>>> require() will not affect applications negatively (there are probably
>>> many
>>>>> more significant overheads we live with today in cordova) -- but I
>>>>> understand why that may not be useful default.
>>>>>
>>>>> In that case, some ideas: (I recall these were proposed previously but
>>> not
>>>>> sure by whom)
>>>>> (a) Bundle tests as a plugin-within-the-plugin as such:
>>>>>   myplugin/
>>>>>     - plugin.xml
>>>>>     - src/...
>>>>>     - www/...
>>>>>     - tests/
>>>>>       - plugin.xml
>>>>>       - www/...
>>>>>
>>>>> Which basically means the plugin tests live in the same repo/branch,
and
>>>>> are fetched as part of "plugin add", but are not moved into platforms/
>>> on
>>>>> cordova prepare by default, thus don't end up in your application (disk
>>>>> and
>>>>> network are cheap, application startup and size costs are not, right?).
>>>>> Then, to run tests, we basically need to iterate all plugins looking
>>> for a
>>>>> nested tests/plugin.xml, and install those.  This can be added to
>>>>> CLI/Plugman, or just be a CLI hook even.
>>>>>
>>>>> (b) add a <js-test-module> or <js-module type="test"> that
is only used
>>> if
>>>>> you run prepare with --test.  Similar to the above, but I think requires
>>>>> more CLI/config file changes, which I'm not a fan of.
>>>>> (c) Just ship tests as a second plugin in a second repo, and document
>>> how
>>>>> to install tests.  Can then perhaps have a <dependency type=tests>.
 I
>>>>> don't like this as much since its basically back to mobile-spec.
>>>>>
>>>>> WDYT?
>>>>>
>>>>> -Michal
>>>>>
>>>>>
>>>>>> On Thu, Jun 19, 2014 at 4:53 PM, Jesse <purplecabbage@gmail.com>
wrote:
>>>>>>
>>>>>> re:
>>>>>>
>>>>>>    Q: What do I do if my plugin tests must have very large assets?
>>>>>>    - A: Don't bundle those assets with your plugin. If you can, have
>>>>> your
>>>>>> ...
>>>>>>
>>>>>>
>>>>>> My concern is I do not want to see tests added to every project that
>>>>> uses a
>>>>>> plugin, even if the assets are not large, there are implications
to
>>>>>> including the test framework + all the tests because they get loaded
>>> and
>>>>>> processed with all of the plugins and will impact load time even
if
>>>>> never
>>>>>> run.
>>>>>>
>>>>>> 99.9% of the time the plugin tests will be used by us the plugin
>>>>>> developers, and not the people who use the plugin in there apps.
>>>>>>
>>>>>> I agree, having the tester install the test harness plugin dependency
>>>>>> themselves is probably a better option, as I see you have wrapped
all
>>>>> tests
>>>>>> inside a exports.defineAutoTests so we don't have to worry about
>>>>>> describe/it/expects not being defined.
>>>>>>
>>>>>>
>>>>>> @purplecabbage
>>>>>> risingj.com
>>>>>>
>>>>>>
>>>>>>> On Thu, Jun 19, 2014 at 1:27 PM, Michal Mocny <mmocny@chromium.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Jesee, the branch is NOT a requirement (I don't think I documented
>>> it
>>>>> as
>>>>>>> such, except in the examples for installing plugins for initial
>>> look).
>>>>>>> Actually, we should delete those stale branches now that we are
>>>>> moving
>>>>>>> up-to-date tests into master.  It was just for early experimentation
>>>>> on
>>>>>> the
>>>>>>> feature.
>>>>>>>
>>>>>>> Jesse, I'm not seeing the benefit of using plugins-tests.xml
or the
>>>>>>> dependency on the test plugin yet, may you elaborate your thoughts?
>>>>>>>
>>>>>>> My hope was that tests are just always installed alongside plugins.
>>>>> If
>>>>>>> that is not a good idea for some particular plugin, say because
it
>>>>> uses
>>>>>>> huge assets, I elaborated my answer in the plugin FAQ (
>>> https://github.com/apache/cordova-labs/blob/cdvtest/cordova-plugin-test-framework/README.md#faq
>>>>>>> ):
>>>>>>> FAQ
>>>>>>>
>>>>>>>    -
>>>>>>>
>>>>>>>    Q: Should I add org.apache.cordova.test-harness as a <dependancy>
>>>>> of
>>>>>> my
>>>>>>>    plugin?
>>>>>>>    - A: No. The end-user should decide if they want to install
the
>>>>> test
>>>>>>>       harness, not your plugin (most users won't).
>>>>>>>    -
>>>>>>>
>>>>>>>    Q: What do I do if my plugin tests must have very large assets?
>>>>>>>    - A: Don't bundle those assets with your plugin. If you can,
have
>>>>> your
>>>>>>>       tests fail gracefully if those assets don't don't exist
>>> (perhaps
>>>>>> log
>>>>>>> a
>>>>>>>       warning, perhaps fail a single asset-checking test, and
skip
>>> the
>>>>>>> rest).
>>>>>>>       Then, ideally download those assets automatically into
local
>>>>>> storage
>>>>>>> the
>>>>>>>       first time tests run. Or create a manual test step to download
>>>>>>> and install
>>>>>>>       assets. As a final alternative, split those test assets
into a
>>>>>>> separate
>>>>>>>       plugin, and instruct users to install that plugin to run
your
>>>>> full
>>>>>>> test
>>>>>>>       suite.
>>>>>>>    -
>>>>>>>
>>>>>>>    Q: Should I ship my app with the test harness plugin installed?
>>>>>>>    - A: Not likely. If you want, you can. Then your app could
even
>>>>> embed
>>>>>> a
>>>>>>>       link to the test page (cdvtests/index.html) from a help
>>> section
>>>>> of
>>>>>>>       your app, to give end users a way to run your test suite
out
>>> in
>>>>>>> the feild.
>>>>>>>       That may help diagnose causes of issues within your app.
>>> Maybe.
>>>>>>>
>>>>>>>
>>>>>>> =============
>>>>>>>
>>>>>>>
>>>>>>> Feel free the debate those answers -- now is certainly the time
--
>>>>> but I
>>>>>>> put a lot of effort to make it super flexible and to not require
>>>>>> depending
>>>>>>> on changes to CLI :P
>>>>>>>
>>>>>>> -Michal
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 19, 2014 at 3:11 PM, Jesse <purplecabbage@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> Sorry I missed providing feedback on this earlier ...
>>>>>>>> Having a deeper look at this, I am not feeling great about
the
>>> extra
>>>>>>>> requirement that every plugin have an additional branch.
>>>>>>>>
>>>>>>>> Several concerns arise :
>>>>>>>> - test branch can be out of sync with master
>>>>>>>> - how do we test a specific version?
>>>>>>>> - tests are not immediately visible when looking at master
>>>>>>>> - differing versions of plugin.xml depending on the branch
>>>>>>>>
>>>>>>>> The majority of the work has been done (thanks Michal!),
and
>>> mostly
>>>>> any
>>>>>>>> suggestions I make will just require moving code and changing
a
>>> few
>>>>>>>> conventions.
>>>>>>>>
>>>>>>>> What if we? :
>>>>>>>> 1. add a plugin-test.xml file which has the exact format
of
>>>>> plugin.xml
>>>>>>>> 2. keep tests/ and plugin-test.xml file in master branch
>>>>>>>> 3. have plugman/cli support an additional flag --test so
we can
>>>>> install
>>>>>>>> like this:
>>>>>>>> cordova plugin add
>>>>>>>> http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git
>>>>>> --test
>>>>>>>> This would mean that in addition to processing the plugin.xml
of
>>> the
>>>>>>>> plugin, we would also process the plugin-test.xml file (
identical
>>>>>>>> processing logic )
>>>>>>>> 4. have all plugin-test.xml files declare a dependency on
>>>>>>>> cordova-plugin-test-framework
>>>>>>>>
>>>>>>>> The above suggestions could also be used in conjuction with
the
>>>>> cordova
>>>>>>> run
>>>>>>>> --tests platform mentioned by Michal, but without the need
to
>>> manage
>>>>>> the
>>>>>>>> switching of branches.
>>>>>>>>
>>>>>>>>
>>>>>>>> @purplecabbage
>>>>>>>> risingj.com
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 17, 2014 at 2:16 PM, Michal Mocny <
>>> mmocny@chromium.org>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Piotr: Actually I'm not sure how running tests in the
harness
>>>>> would
>>>>>>> work,
>>>>>>>>> since the path to the resource may be different.  However,
in
>>>>>> general,
>>>>>>>> with
>>>>>>>>> development using the harness you aren't making any changes
to
>>>>>> plugins.
>>>>>>>>> The whole point is for app developers who want to modify
only
>>> web
>>>>>>>>> application bits and not deal with native compiles.
>>>>>>>>>
>>>>>>>>> In theory the app harness could support working on the
>>> js-modules
>>>>> of
>>>>>>>>> plugins, but that sounds like a really niche idea.  I'd
not be
>>>>>> opposed
>>>>>>> to
>>>>>>>>> someone working on it but I'm not sure you'll have luck
finding
>>>>>>>> volunteers.
>>>>>>>>>
>>>>>>>>> -Michal
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 17, 2014 at 5:13 PM, Michal Mocny <
>>>>> mmocny@chromium.org>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> At the time I went through my design iterations I
just didn't
>>>>> want
>>>>>> to
>>>>>>>>>> necessarily depend on cordova tooling changes / documentation.
>>>>> In
>>>>>>>> other
>>>>>>>>>> words, someone else may have a different strategy
for
>>> testing..
>>>>>>>>>>
>>>>>>>>>> My personal opinion would be have the test plugin
ship with a
>>>>>> plugin
>>>>>>>> hook
>>>>>>>>>> (those are in, right? or at least on their way),
that will
>>>>>>>> automatically
>>>>>>>>>> update the start page if you pass a flag to run command.
 Ie,
>>>>> in an
>>>>>>>> ideal
>>>>>>>>>> world:  `cordova run --tests` runs a plugin hook
passing in
>>>>> --tests
>>>>>>>> flag
>>>>>>>>>> which changes the start page, in a way that isn't
overwritten
>>> by
>>>>>> the
>>>>>>>>>> top-level config.xml.
>>>>>>>>>>
>>>>>>>>>> My 2 cents, since I don't want "our way" of testing
mobile
>>> spec
>>>>> to
>>>>>> be
>>>>>>>>> "the
>>>>>>>>>> only way" to test.   Frameworks and opinions on testing
>>> change.
>>>>>>>>>>
>>>>>>>>>> -Michal
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 17, 2014 at 4:33 PM, Piotr Zalewa <
>>>>> pzalewa@mozilla.com
>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> One thing more - it would be great if user could
create a
>>> test
>>>>>> using
>>>>>>>>> test
>>>>>>>>>>> harness app as well. Is it also considered?
>>>>>>>>>>>
>>>>>>>>>>> Dnia Tue Jun 17 13:27:22 2014 Martin Gonzalez
pisze:
>>>>>>>>>>>
>>>>>>>>>>> It would be a nice to have in the cli, aimed
to just setup
>>> the
>>>>>>> right
>>>>>>>>> path
>>>>>>>>>>>> in the config.xml, maybe along with an another
argument to
>>>>> build,
>>>>>>>>>>>> run/emulate as well.
>>>>>>>>>>>> It sounds great.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2014-06-17 15:21 GMT-05:00 Piotr Zalewa <
>>> pzalewa@mozilla.com
>>>>>> :
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks Martin,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Has it been considered to create a separate
command
>>>>> "testrun" or
>>>>>>>>> similar
>>>>>>>>>>>>> which would remove the need to edit the
config.xml?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dnia Tue Jun 17 11:58:33 2014 Michal
Mocny pisze:
>>>>>>>>>>>>>
>>>>>>>>>>>>>   Martin, thanks for posting those links.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And I'll look into the INFRA tickets
I need to file to set
>>>>> up a
>>>>>>>> repo
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> that plugin, since its ready to come
out of labs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jun 17, 2014 at 2:06 PM,
Martin Gonzalez <
>>>>>>>>>>>>>> martin.c.glez.glez@gmail.com>
wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>   This is the Cordova Plugin Test
Framework readme.md,
>>> you
>>>>> can
>>>>>>>> catch
>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> the functionality by reading
some of the content:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Repository:
>>>>>>>>>>>>>>> https://github.com/apache/cordova-labs
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Docs:
>>>>> https://github.com/apache/cordova-labs/blob/master/README.md
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://github.com/apache/cordova-labs/blob/cdvtest/
>>>>>>>>>>>>>>> cordova-plugin-test-framework/README.md
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2014-06-17 12:56 GMT-05:00 Piotr
Zalewa <
>>>>> pzalewa@mozilla.com
>>>>>>> :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>   a documentation explaining
how it's gonna work
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Dnia Tue Jun 17 10:51:58
2014 Michal Mocny pisze:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>    What do you mean?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jun 17, 2014
at 1:50 PM, Piotr Zalewa <
>>>>>>>>> pzalewa@mozilla.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>    Is there any predev
document?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Dnia Mon Jun 16 18:30:46
2014 Andrew Grieve pisze:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>     Yeah, really
exciting. Thanks for taking this on.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jun 16,
2014 at 3:42 PM, Michal Mocny <
>>>>>>>>>>>>>>>>>>> mmocny@chromium.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>     Fantastic!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'll try to keep
an eye out on the PR's, and please
>>>>> ping
>>>>>> me
>>>>>>>> if
>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> like any
help.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -Michal
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Jun
16, 2014 at 3:25 PM, Marcel Kinard <
>>>>>>>>>>>>>>>>>>>> cmarcelk@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>     Hi, after
some discussions here with IBM
>>>>> management,
>>>>>>>> we’re
>>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>   bring
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> in a
couple extra interns for a week to jumpstart
>>> the
>>>>>>>>> migration
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> tests out of mobile-spec into
the new plugin test
>>>>> framework.
>>>>>>>> Staci
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cooper
>>>>>>>>>>>>>>>>>>>>> will
be leading this effort, and Martin Gonzalez
>>> will
>>>>>> be a
>>>>>>>>> part
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So if you see a bunch of
pull requests, this is what it
>>>>> is
>>>>>>> for.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> We’ll
>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the interns to submit
an ICLA asap.
>>>>>>>>>>>>>>>>>>>>     --
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Piotr Zalewa
>>>>>>>>>>>>>>>>>> Mozilla
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>    --
>>>>>>>>>>>>>>>> Piotr Zalewa
>>>>>>>>>>>>>>>> Mozilla
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Martin Gonzalez
>>>>>>>>>>>>>> --
>>>>>>>>>>>>> Piotr Zalewa
>>>>>>>>>>>>> Mozilla
>>>>>>>>>>> --
>>>>>>>>>>> Piotr Zalewa
>>>>>>>>>>> Mozilla
>>>

--
Piotr Zalewa
Mozilla

Mime
View raw message