cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Bowser <bows...@gmail.com>
Subject Re: We need better contributions from new people
Date Wed, 12 Jun 2013 22:10:14 GMT
The problem with this approach is that we shouldn't rely on just
mobile-spec for our testing. In this example, I wasn't able to fully
reproduce what he was doing because the person who filed it refused to
provide me with a project.  That being said, I was able to find that I
was getting weird TIMEOUT errors with the code due to it going through
the wrong method.  While on my devices, it was a really minor error in
logcat that would be easily ignored, it apparently causes meltdowns on
the emulator.

The fact is that on Android, if you're doing changes to the core, you
should be running the JUnit tests incldued with the project.  These
tests are completely independent of mobile-spec and catch a majority
of the native use cases that we currently have running.  mobile-spec
is the cross-platform test suite, and every platform should have its
own native test suite to test its internals to make sure we're not
totally screwing things up.  If there's an error from Cordova that
appears after running it, chances are that something is wrong, even if
the tests pass.  Remember that for certain bugs, like bugs where you
exit the app, which this one was, it's really hard to test those even
with JUnit.

As far as this being a recurring problem, I do think that it is
starting recurring problem, since our users are saying that they
haven't seen a quality release of Cordova-Android since 2.4.  That
means that we broke something in the past four months that meant that
they couldn't upgrade.  Some of it was stuff we couldn't avoid, like
the WebSQL vs WebStorage choice, or deprecations of Plugins.java, but
most of it was stuff like this and the Splashscreen error, where we'd
accept a patch without testing.

So, please write more JUnit tests as well as mobile-spec.  mobile-spec
is not a silver bullet for testing on Cordova, since it really only
tests Cordova JS and the API.

On Wed, Jun 12, 2013 at 2:07 PM, Marcel Kinard <cmarcelk@gmail.com> wrote:
> +1 to Ian's comments.
>
> How about if:
> - the developer (contributor or committer) is responsible to test their own code and
correct any problems before a pull request is submitted (contributor authored) or it lands
in the stream (committer authored). The testing includes both verifying the function they
added/touched, plus running mobile-spec to verify there are no regressions.
> - if a pull request, the committer should sanity check the contributor's code.
> - if a pull request, the committer should see that the contributor tested it. Perhaps
that could be a simple comment in Jira, similar to how Mike and Lisa worked as contributors
here: https://issues.apache.org/jira/browse/CB-3521 .
>
> I find it odd that "Processing Pull Requests" in http://wiki.apache.org/cordova/CommitterWorkflow
doesn't mention anything about testing or sanity checking / reviewing what comes from the
contributor. If there is consensus, I can add the verbage above to "Processing Pull Requests".
(Hmm, shouldn't there also be something there about doc for external behaviors / interfaces
/ requirements?)
>
> Based on other commercial projects I've seen, and how aggressive you want to be, I'd
also be tempted to suggest:
> - every new function and defect fix should include an automated test (where reasonably
feasible) that lands in mobile-spec at the same time as the new function / fix. What this
does over time is to build up coverage in the automated test suite that gets a lot better
at catching breakages / regressions. Yes, this takes time to do, but when mobile-spec is run,
you've got confidence.
>
> -- Marcel Kinard

Mime
View raw message