cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Grebnov (Akvelon)" <v-seg...@microsoft.com>
Subject RE: Mobile spec tests and exclusion list
Date Sat, 26 Oct 2013 10:41:39 GMT
Hi Michael,

Agree. But taking into account having a way to run all the tests (including ones w/ user interaction)
is very useful for Windows Phone I propose the following
1. No changes for non-WP platforms
2. For WP
  a) Use the following condition for the tests which require user interaction  
    define(..., function(...) {
      if (isWP8 && !runAll) return;
      expect(...);
      ...
    })
  b) Current autotests will run w/o runAll option so won't require user interaction
  c) Add 'Run All Tests (Extended)' option specifically for WP8 where we will have runAll
== true 

Motivation:
1. I don't think we should move such tests to manual tests for WP only to be consistent with
other platforms - we actually test api call and check result
2. By default all tests will run w/o any user interaction
3. We will have an option to quickly check all api before release via Run All Tests (Extended).
In other case we should have special information how to check all the api and not to forget
to run such special tests.

Thx!
Sergey
-----Original Message-----
From: mmocny@google.com [mailto:mmocny@google.com] On Behalf Of Michal Mocny
Sent: Saturday, October 26, 2013 4:12 AM
To: dev
Subject: Re: Mobile spec tests and exclusion list

Auto tests should run automatically without intervention.  If user actions is needed for test
to pass, we should call that something different (manual tests have been used).

I think some varient of #3 is fine, this isn't a common problem.  I wouldn't even test for
Medic specifically, since I want my auto tests to run automatically even when testing by hand.

define(..., function(...) {
  if (isWP8) return;
  expect(...);
  ...
})

-Michal


On Fri, Oct 25, 2013 at 4:37 PM, Sergey Grebnov (Akvelon) < v-segreb@microsoft.com>
wrote:

> Mobile spec autotests include tests which on some platforms require 
> user interaction to complete. For example, contact save api on Windows 
> Phone requires user to manually click on save  button. This prevents 
> the tests to be run as part of Medic test automation since test  app 
> just hangs on such api calls.
>
> Is Windows Phone special or there are similar problem on other platforms?
>
> I'm thinking about the following possible approaches:
> #1 Ad-hoc solution to Medic - replacing some test files as part of 
> Medic functionality (some additional wp specific build step).
> #2 Extending mobile spec functionality- adding something like tests 
> exclusion config where you can define test ids (or even the whole api) 
> to be skipped. Such exclusion list could be generated on the fly and 
> put to the app before starting tests.
> #3 If there are only few such tests we can probably add check for the 
> current platform to determine whether to include the test. For example:
> if(!(window.MedicTestRunner && isWP8)) {testDefinition}  Or the same 
> way but inside the test to fail gracefully.
>
> Thoughts?
>
> Thx!
> Sergey
>

Mime
View raw message