couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dale Harvey <d...@arandomurl.com>
Subject Re: Getting libraries to test RCs
Date Fri, 02 Sep 2016 10:14:35 GMT
In PouchDB we can look into a workaround that uses random names only when
the tests are run against Couch 2.0, however I would really like to make
sure that a database not being fully deleted when we get a successful
confirmation of deletion is considered a bug, it has impacts beyond the
test suite, its really hard to create a reliable system when there is no
way for you to be certain when a database is deleted.

Will found it easiest to reproduce this using concurrent scripts but would
like to clarify that Pouch doesnt run the test suite in parallel, this bug
can be hit by doing CREATE -> DELETE -> CREATE, its extremely hard to nail
down and reproduce (the similiar bug in PouchDB took many attempts +
months). I will take a look at seeing if I can make an easier and clearer
steps to reproduce.

On 2 September 2016 at 11:01, Jan Lehnardt <jan@apache.org> wrote:

>
> > On 02 Sep 2016, at 11:58, Will Holley <willholley@gmail.com> wrote:
> >
> > Jan - I can understand that being the case in a clustered setup with
> > distributed shard maps but shouldn't n=1 mitigate that?
>
> n=1 still does q=8 (8 shards per node) and the software makes
> noconsistency guarantees whatsoever.
>
> n=1 && q=1 might work as a side-effect, but not sure how that is useful
> for reliable tests :)
>
> Best
> Jan
> --
>
>
> >
> > On 2 September 2016 at 10:53, Jan Lehnardt <jan@apache.org> wrote:
> >>
> >>> On 02 Sep 2016, at 11:45, Dale Harvey <dale@arandomurl.com> wrote:
> >>>
> >>> In PouchDB we used to generate unique database names for tests,
> however we
> >>> removed it for serveral reasons, one large reason being it indicates a
> race
> >>> condition in critical code if we cannot reliably create -> delete ->
> create
> >>> the same database (we have uncovered and fixed a lot of bugs in
> PouchDB due
> >>> to this). While its not my call how to prioritise those bugs, I really
> do
> >>> not think we should be closing what are fairly serious bugs because it
> >>> wasnt inconvenient to workaround them in the couch test suite.
> >>
> >> It’s just that a CouchDB 2.0 cluster is an AP system, and recreating
> databases
> >> in quick succession reliably basically requires a CA system and that’s
> not what can do easily.
> >>
> >> (I hope I got the CAP letters right, but I think it is clear what I
> mean)
> >>
> >> That is, maybe we skip those tests when run against a CouchDB 2.0
> endpoint and keep them for PouchDB?
> >>
> >> Best
> >> Jan
> >> --
> >>
> >>
> >>>
> >>> On 2 September 2016 at 10:31, Joan Touzet <wohali@apache.org> wrote:
> >>>
> >>>> Hi Nolan, Will:
> >>>>
> >>>> A further update from looking deeper with @janl. It appears that we
> >>>> have a pending fix for COUCHDB-3017 and we'll work on getting that
> >>>> merged before 2.0.
> >>>>
> >>>> COUCHDB-3034 is a WONTFIX. FYI in CouchDB itself we changed all of
> >>>> our tests to use unique database names. I'll update the bug myself
> >>>> shortly.
> >>>>
> >>>> -Joan
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Joan Touzet" <wohali@apache.org>
> >>>>> To: dev@couchdb.apache.org
> >>>>> Sent: Friday, September 2, 2016 5:15:00 AM
> >>>>> Subject: Re: Getting libraries to test RCs
> >>>>>
> >>>>> Hi Will,
> >>>>>
> >>>>> Neither of these are currently tagged as blocking issues for CouchDB
> >>>>> 2.0, only major priority. If you want to flag them as such, this
is
> >>>>> your last chance, and even still, there's no guarantee fixes for
them
> >>>>> will hit 2.0.
> >>>>>
> >>>>> Erlangers, is there any chance of at least triaging these today?
> >>>>>
> >>>>> -Joan
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Will Holley" <willholley@gmail.com>
> >>>>>> To: dev@couchdb.apache.org, "Joan Touzet" <wohali@apache.org>
> >>>>>> Sent: Friday, September 2, 2016 4:43:48 AM
> >>>>>> Subject: Re: Getting libraries to test RCs
> >>>>>>
> >>>>>> Assuming nothing's changed in the last few weeks, there are
2
> >>>>>> issues
> >>>>>> which cause the PouchDB tests to fail against master: COUCHDB-3017
> >>>>>> and
> >>>>>> COUCHDB-3034.
> >>>>>>
> >>>>>> Both could be addressed in the test suite by using different
> >>>>>> database
> >>>>>> names for each test, but that's quite a disruptive change.
> >>>>>>
> >>>>>> On 2 September 2016 at 03:15, Joan Touzet <wohali@apache.org>
> >>>>>> wrote:
> >>>>>>> Hi Nolan, you state that it's 'failing for known reasons.'
Is
> >>>>>>> that
> >>>>>>> reasons in PouchDB or anything you need to push back on
us? We'd
> >>>>>>> like
> >>>>>>> to know ASAP as we're very, very close to releasing 2.0
now.
> >>>>>>>
> >>>>>>> I have zero PouchDB knowledge so I'm hoping you can give
us a
> >>>>>>> short
> >>>>>>> summary of what you think is wrong.
> >>>>>>>
> >>>>>>> All the best,
> >>>>>>> Joan
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>> From: "Nolan Lawson" <nolan@nolanlawson.com>
> >>>>>>>> To: dev@couchdb.apache.org
> >>>>>>>> Sent: Thursday, September 1, 2016 7:56:42 PM
> >>>>>>>> Subject: Re: Getting libraries to test RCs
> >>>>>>>>
> >>>>>>>> We have been testing CouchDB master in PouchDB for months
now,
> >>>>>>>> but
> >>>>>>>> as
> >>>>>>>> an allowed failure because I believe it’s failing
for known
> >>>>>>>> reasons.
> >>>>>>>> We test both using Node.js and the browser.
> >>>>>>>>
> >>>>>>>> Node: https://travis-ci.org/pouchdb/pouchdb/jobs/156198210
> >>>>>>>> Browser: https://travis-ci.org/pouchdb/pouchdb/jobs/156198211
> >>>>>>>>
> >>>>>>>> For anyone who wants to run the Pouch test suite against
> >>>>>>>> CouchDB,
> >>>>>>>> it’s just:
> >>>>>>>>
> >>>>>>>> git clone https://github.com/pouchdb/pouchdb.git
> >>>>>>>> cd pouchdb
> >>>>>>>> npm I
> >>>>>>>> COUCH_HOST=http://localhost:5984 BAIL=0 npm t
> >>>>>>>>
> >>>>>>>> BAIL=0 will tell it to run the full test suite and not
stop on
> >>>>>>>> any
> >>>>>>>> failures. That way you can inspect the failures and
see if
> >>>>>>>> they’re
> >>>>>>>> serious or not.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Nolan
> >>>>>>>>
> >>>>>>>>> On Aug 29, 2016, at 12:15 PM, Jan Lehnardt <jan@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Anyone on this list who could help with this? The
work items
> >>>>>>>>> are
> >>>>>>>>> fairly self-explanatory and not very big individually
<3
> >>>>>>>>>
> >>>>>>>>> Best
> >>>>>>>>> Jan
> >>>>>>>>> --
> >>>>>>>>>
> >>>>>>>>>> On 10 Aug 2016, at 09:37, Jan Lehnardt <jan@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hey everyone,
> >>>>>>>>>>
> >>>>>>>>>> from Joan’s excellent blog post about testing
Release
> >>>>>>>>>> Candidates:
> >>>>>>>>>>
> >>>>>>>>>>> To our valued CouchDB application and library
developers:
> >>>>>>>>>>> please,
> >>>>>>>>>>> please run your software against each of
the options below.
> >>>>>>>>>>
> >>>>>>>>>> — https://blog.couchdb.org/2016/08/08/release-candidates/
> >>>>>>>>>>
> >>>>>>>>>> I think we can be a little more proactive about
this for
> >>>>>>>>>> CouchDB
> >>>>>>>>>> client libraries: let’s open issues on all
the
> >>>>>>>>>> CouchDB-compatible
> >>>>>>>>>> client software we care about to test an RC.
> >>>>>>>>>>
> >>>>>>>>>> Since there are a lot of projects, and we don’t
necessarily
> >>>>>>>>>> know
> >>>>>>>>>> which one we “care” about, we should try
to be clever about
> >>>>>>>>>> it.
> >>>>>>>>>>
> >>>>>>>>>> Maybe something like this can work:
> >>>>>>>>>>
> >>>>>>>>>> 1. We prepare an issue text explaining the thing:
Heya,
> >>>>>>>>>> CouchDB
> >>>>>>>>>> team here, major new version coming up, you
should test it
> >>>>>>>>>> like
> >>>>>>>>>> so: <include instructions to test against
a 3-node cluster.
> >>>>>>>>>> Maybe
> >>>>>>>>>> even provide a cluster to do this, or Cloudant
can sponsor
> >>>>>>>>>> something?
> >>>>>>>>>>
> >>>>>>>>>> 2. Post this message with a call to action on
user@c.a.o, the
> >>>>>>>>>> weekly news, and our other (social) media channels.
> >>>>>>>>>>
> >>>>>>>>>> 3. Ask people who submitted an issue to report
back with a
> >>>>>>>>>> link.
> >>>>>>>>>>
> >>>>>>>>>> 4. Collect the link in an issue or JIRA (this
could be done
> >>>>>>>>>> in
> >>>>>>>>>> 3.,
> >>>>>>>>>> but then everybody needs to be added to the
wiki write group,
> >>>>>>>>>> and
> >>>>>>>>>> that’s just extra overhead we don’t need).
Maybe we borrow a
> >>>>>>>>>> gist
> >>>>>>>>>> for this, or a Google doc.
> >>>>>>>>>>
> >>>>>>>>>> That way we encourage client software to check
out RCs and we
> >>>>>>>>>> can
> >>>>>>>>>> keep track, while the community helps to select
which
> >>>>>>>>>> software
> >>>>>>>>>> to
> >>>>>>>>>> encourage to test 2.0 compat, and helps spread
the word and
> >>>>>>>>>> the
> >>>>>>>>>> burden is not left with just a few folks.
> >>>>>>>>>>
> >>>>>>>>>> What do you think?
> >>>>>>>>>>
> >>>>>>>>>> Best
> >>>>>>>>>> Jan
> >>>>>>>>>> --
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Professional Support for Apache CouchDB:
> >>>>>>>>> https://neighbourhood.ie/couchdb-support/
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >> --
> >> Professional Support for Apache CouchDB:
> >> https://neighbourhood.ie/couchdb-support/
> >>
>
> --
> Professional Support for Apache CouchDB:
> https://neighbourhood.ie/couchdb-support/
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message