cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Mocny <mmo...@chromium.org>
Subject Re: Mobile spec tests and exclusion list
Date Sat, 26 Oct 2013 22:24:51 GMT
We have a proposal and prototype on the table right now for re-working
tests to ship with plugins, defined according to auto and manual tests.

To accomplish what you ask for would require a specialized testing app that
simply runs both at the same time. (this wouldn't be the default, but would
be easy to make).

Thus, I think the tests shouldn't be modified (its hard to state at test
definition time in which fashion they should be used), the test runner
should.  This wont solve the problem today, but perhaps in about a month it
could.

-Micha


On Sat, Oct 26, 2013 at 6:41 AM, Sergey Grebnov (Akvelon) <
v-segreb@microsoft.com> wrote:

> Hi Michael,
>
> Agree. But taking into account having a way to run all the tests
> (including ones w/ user interaction) is very useful for Windows Phone I
> propose the following
> 1. No changes for non-WP platforms
> 2. For WP
>   a) Use the following condition for the tests which require user
> interaction
>     define(..., function(...) {
>       if (isWP8 && !runAll) return;
>       expect(...);
>       ...
>     })
>   b) Current autotests will run w/o runAll option so won't require user
> interaction
>   c) Add 'Run All Tests (Extended)' option specifically for WP8 where we
> will have runAll == true
>
> Motivation:
> 1. I don't think we should move such tests to manual tests for WP only to
> be consistent with other platforms - we actually test api call and check
> result
> 2. By default all tests will run w/o any user interaction
> 3. We will have an option to quickly check all api before release via Run
> All Tests (Extended). In other case we should have special information how
> to check all the api and not to forget to run such special tests.
>
> Thx!
> Sergey
> -----Original Message-----
> From: mmocny@google.com [mailto:mmocny@google.com] On Behalf Of Michal
> Mocny
> Sent: Saturday, October 26, 2013 4:12 AM
> To: dev
> Subject: Re: Mobile spec tests and exclusion list
>
> Auto tests should run automatically without intervention.  If user actions
> is needed for test to pass, we should call that something different (manual
> tests have been used).
>
> I think some varient of #3 is fine, this isn't a common problem.  I
> wouldn't even test for Medic specifically, since I want my auto tests to
> run automatically even when testing by hand.
>
> define(..., function(...) {
>   if (isWP8) return;
>   expect(...);
>   ...
> })
>
> -Michal
>
>
> On Fri, Oct 25, 2013 at 4:37 PM, Sergey Grebnov (Akvelon) <
> v-segreb@microsoft.com> wrote:
>
> > Mobile spec autotests include tests which on some platforms require
> > user interaction to complete. For example, contact save api on Windows
> > Phone requires user to manually click on save  button. This prevents
> > the tests to be run as part of Medic test automation since test  app
> > just hangs on such api calls.
> >
> > Is Windows Phone special or there are similar problem on other platforms?
> >
> > I'm thinking about the following possible approaches:
> > #1 Ad-hoc solution to Medic - replacing some test files as part of
> > Medic functionality (some additional wp specific build step).
> > #2 Extending mobile spec functionality- adding something like tests
> > exclusion config where you can define test ids (or even the whole api)
> > to be skipped. Such exclusion list could be generated on the fly and
> > put to the app before starting tests.
> > #3 If there are only few such tests we can probably add check for the
> > current platform to determine whether to include the test. For example:
> > if(!(window.MedicTestRunner && isWP8)) {testDefinition}  Or the same
> > way but inside the test to fail gracefully.
> >
> > Thoughts?
> >
> > Thx!
> > Sergey
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message