incubator-callback-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filip Maj <...@adobe.com>
Subject Re: tagging 1.5rc friday
Date Mon, 27 Feb 2012 21:50:35 GMT
Testing process: this is important :)

Let's hash that out in this thread, and document over at
http://wiki.apache.org/cordova/RunningTests

Here's my version, please add anything that you think I'm missing. For me
it's as simple as:

1. Run the qunit tests in mobile-spec. This is a baseline. I have never
seen it pass 100% but below 50 tests failing (less than 10%) seems to be
what the average across platforms is. Our goal is to reduce this as much
as possible.
2. Run the "manual" tests in mobile-spec. These are the individual
links/pages showing up when you run mobile-spec.

I would like to see the manual tests refined a bit more to have expected
behaviour mapped out more obviously. Something like:
  - you run the accelerometer manual tests.
  - these tests ask you to lie the device flat on a table and hit a
button. We assert that the various x/y/z accel values line up to what we
expect.
  - do the same for various other device positions.
  - rinse and repeat for all of our APIs.

I had Becky express concerns that mobile-spec is not a complete test suite
and "needs improvement." Please, if you feel this is the case, it is
*extremely* important to commit patches to the test suite! It is our only
tool that allows us to say with confidence "we are ok to release."

Moving forward, with cordova-js lined up as dropping in the next few
releases, as well as the embeddable webview + rename changes coming into
the native side on all of our platform implementations, the test suite
needs to be our go-to method for determining what works and what doesn't.
If you feel it is lacking: patch it.

Let's get this process nailed down in our wiki and in this thread so our
next release goes smoother.

On 12-02-27 1:35 PM, "Simon MacDonald" <simon.macdonald@gmail.com> wrote:

>Well running mobile spec is only part of the regression I usually do.
>Typically I'll run the autotests with the previous version then again with
>the latest version and find the differences. By doing that this release I
>was able to fix the issue with the DirectoryReader on Android. Then I'll
>run through the manual tests in mobile spec to make sure that things are
>still working. Then I start going through the examples on
>docs.phonegap.combecause it is really embarrassing when the sample
>code in our docs doesn't
>run.
>
>I've been doing this for a number of releases now and it helps me get a
>feel for how stable we are. With all the moving parts for 1.5 I don't feel
>like we are as stable as 1.4 and don't think we should tag tomorrow as
>there are some more things I want to look at.
>
>Simon Mac Donald
>http://hi.im/simonmacdonald
>
>
>On Mon, Feb 27, 2012 at 4:30 PM, Joe Bowser <bowserj@gmail.com> wrote:
>
>> What regression process are you using?  I've personally have never seen
>> mobile-spec pass on Android, so I'm curious as to what the criteria is.
>>
>> Joe
>>
>> On Mon, Feb 27, 2012 at 1:24 PM, Simon MacDonald
>> <simon.macdonald@gmail.com>wrote:
>>
>> > I do not feel comfortable tagging 1.5.0 tomorrow. I'm still finding
>>bugs
>> in
>> > regression.
>> >
>> > Simon Mac Donald
>> > http://hi.im/simonmacdonald
>> >
>> >
>> > On Mon, Feb 27, 2012 at 4:20 PM, Joe Bowser <bowserj@gmail.com> wrote:
>> >
>> > > JIRA says that this is supposed to happen tomorrow.
>> > >
>> > > On Mon, Feb 27, 2012 at 1:18 PM, Simon MacDonald
>> > > <simon.macdonald@gmail.com>wrote:
>> > >
>> > > > Wait, when are we supposed to be releasing 1.5?
>> > > >
>> > > > Simon Mac Donald
>> > > > http://hi.im/simonmacdonald
>> > > >
>> > > >
>> > > > On Mon, Feb 27, 2012 at 4:10 PM, Steven Gill
>><stevengill97@gmail.com
>> >
>> > > > wrote:
>> > > >
>> > > > > I haven't pulled the release candidate together yet, so feel
>>free
>> to
>> > > > retag.
>> > > > >
>> > > > > -Steve
>> > > > >
>> > > > > On Mon, Feb 27, 2012 at 12:58 PM, Becky Gibson <
>> > gibson.becky@gmail.com
>> > > > > >wrote:
>> > > > >
>> > > > > > Has anything been done with the build?  I would like to
add
>>the
>> > > > > > <type>Entry.toURL() api and deprecate to.URI() into
the
>>release
>> > > > > candidate.
>> > > > > >  Is ready to go, just need to retag if this change is to
go
>>in.
>> > > > > >
>> > > > > > -becky
>> > > > > >
>> > > > > >
>> > > > > > On Mon, Feb 27, 2012 at 12:35 PM, Becky Gibson <
>> > > gibson.becky@gmail.com
>> > > > > > >wrote:
>> > > > > >
>> > > > > > > iOS is tagged for 1.5.0rc1
>> > > > > > >
>> > > > > > >
>> > > > > > > On Sat, Feb 25, 2012 at 12:31 AM, Simon MacDonald <
>> > > > > > > simon.macdonald@gmail.com> wrote:
>> > > > > > >
>> > > > > > >> One test case was using escapeURL when it should
have used
>> > > escapeURI
>> > > > > so
>> > > > > > I
>> > > > > > >> fixed it and checked it in. Also, in common-js
there was a
>>bug
>> > in
>> > > > the
>> > > > > > >> readEntries method where it was referencing this.fullPath
>>when
>> > it
>> > > > > should
>> > > > > > >> have been this.path. I fixed that as well.
>> > > > > > >>
>> > > > > > >> I've regenerated the cordova.android.js file and
checked
>>it in
>> > > along
>> > > > > > with
>> > > > > > >> a
>> > > > > > >> version bump for Android to 1.5.0rc1.
>> > > > > > >>
>> > > > > > >> Simon Mac Donald
>> > > > > > >> http://hi.im/simonmacdonald
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> On Fri, Feb 24, 2012 at 6:53 PM, Joe Bowser <
>> bowserj@gmail.com>
>> > > > > wrote:
>> > > > > > >>
>> > > > > > >> > Which test case?
>> > > > > > >> >
>> > > > > > >> > On Fri, Feb 24, 2012 at 12:23 PM, Simon MacDonald
<
>> > > > > > >> > simon.macdonald@gmail.com
>> > > > > > >> > > wrote:
>> > > > > > >> >
>> > > > > > >> > > I will not be tagging Android today.
There are a few
>> > > previously
>> > > > > > >> passing
>> > > > > > >> > > test cases in mobile-spec that are now
failing. Will
>>tag
>> > once
>> > > > the
>> > > > > > >> problem
>> > > > > > >> > > is resolved.
>> > > > > > >> > >
>> > > > > > >> > > Simon Mac Donald
>> > > > > > >> > > http://hi.im/simonmacdonald
>> > > > > > >> > >
>> > > > > > >> > >
>> > > > > > >> > > On Fri, Feb 24, 2012 at 1:54 PM, Steve
Gill <
>> > > > > stevengill97@gmail.com
>> > > > > > >
>> > > > > > >> > > wrote:
>> > > > > > >> > >
>> > > > > > >> > > > Hey guys,
>> > > > > > >> > > >
>> > > > > > >> > > > Lets tag 1.5.0rc1 today. Post here
once your repo is
>> > tagged.
>> > > > > > >> > > >
>> > > > > > >> > > > -Steve
>> > > > > > >> > > >
>> > > > > > >> > > >
>> > > > > > >> > > >
>> > > > > > >> > > >
>> > > > > > >> > > > On Feb 22, 2012, at 9:31 AM, Brian
LeRoux
>><b@brian.io>
>> > > wrote:
>> > > > > > >> > > >
>> > > > > > >> > > > > notes from call this am
>> > > > > > >> > > > >
>> > > > > > >> > > > > - aim to release 1.5 on tues
>> > > > > > >> > > > > - cordova js unlikely for wp,
ios, blackberry
>> > > > > > >> > > > > - cordova rename from phonegap
on track
>> > > > > > >> > > > > - coho release tool now does
gpg signing: need
>>input
>> > from
>> > > > > > mentors
>> > > > > > >> on
>> > > > > > >> > > > > cutting official apache release
>> > > > > > >> > > > > - plugins are breaking: create
a wiki page about
>> > updating
>> > > > > ---but
>> > > > > > >> > > > > plugins have bigger problems
which we aim to
>>address
>> > > > sometime
>> > > > > > >> between
>> > > > > > >> > > > > now and 2.x
>> > > > > > >> > > >
>> > > > > > >> > >
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>


Mime
View raw message