cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Bowser <bows...@gmail.com>
Subject Re: Understanding the intent of the test suites and how they fit together
Date Thu, 04 Jul 2013 17:29:15 GMT
On Thu, Jul 4, 2013 at 9:20 AM, Andrew Grieve <agrieve@chromium.org> wrote:
> Comments inline.

As are mine.

> On Thu, Jul 4, 2013 at 3:27 AM, Smith, Peter <peters@fast.au.fujitsu.com>wrote:
>
>> Hi devs,
>>
>>
>>
>> I would like to understand better about the actual intent of the various
>> cordova test suites for v2.x.

So, this is for 2.x, not for 3.x.
>>
>> cordova-android tests
>>
>> * Tests are split into manual and JUnit-style tests
>>

False.  The manual tests are there only because we haven't automated
all the tests.  The old manual tests are run by the automatic tests.

>> * There are about 20 of each. (Seems very small number - coverage of
>> Java classes is only about 10%).

We test the API through mobile-spec, so we don't consider these tests
as high a priority.  I noticed that you've been creating a lot of
low-priority issues on the tracker, which I actually don't consider
too helpful, since you already know the answer to over half the
questions that you're asking.

>>
>> * Most JUnit test cases seem focussed on behaviour of the
>> CordovaWebView. It looks like lots of Java code does not get touched by
>> test cases at all.
>>
> Yep, we'd like this suite to be far more extensive.
>

That's because these tests were written back when I developed the
CordovaWebView back when Cordova 2.0 was implemented.  Bryce Curtis
wrote the original manual tests, and I worked through automating them
because I don't believe that manual tests are worth anything, since
manual tests are never run and tests that aren't run are useless
tests.

>>
>> * Manual tests suffer lots of fails in my environment - so much so that
>> it's not even clear if these manual tests are expected to work at all,
>> or are they just a bunch of ad-hoc test fragments which were useful once
>> but have since fallen on hard times
>>

Again, manual tests only exist to be automated.  If a test can't be
automated, it should be removed and put elsewhere.

>
> The intention is that all manual ones are run for each release, and that
> they should pass.
>

In reality, nobody runs manual tests.  This isn't unique to Cordova,
it's a fact in all development.  This is why so much effort is put
into automation.  As far as I know, Android is the only platform with
a tests directory at all.

Honestly, I curious what your intent is with this line of questioning.
 Since this is an open source project, if you think that this testing
is lacking, you can always submit more tests.  Patches are always
welcome.

Mime
View raw message