From dev-return-9233-apmail-couchdb-dev-archive=couchdb.apache.org@couchdb.apache.org Sun Mar 21 17:16:43 2010 Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 80274 invoked from network); 21 Mar 2010 17:16:42 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 21 Mar 2010 17:16:42 -0000 Received: (qmail 97259 invoked by uid 500); 21 Mar 2010 17:16:41 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 97225 invoked by uid 500); 21 Mar 2010 17:16:41 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 97217 invoked by uid 99); 21 Mar 2010 17:16:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Mar 2010 17:16:41 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=AWL,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [80.244.253.218] (HELO mail.traeumt.net) (80.244.253.218) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Mar 2010 17:16:34 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.traeumt.net (Postfix) with ESMTP id A3C731B52D for ; Sun, 21 Mar 2010 18:16:10 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mail.g3th.net Received: from unknown by localhost (amavisd-new, unix socket) id l7WSgl8lJniT for ; Sun, 21 Mar 2010 18:16:09 +0100 (CET) Received: from [10.0.2.6] (rrcs-97-77-198-17.sw.biz.rr.com [97.77.198.17]) (authenticated) by mail.traeumt.net (amavisd-milter) (authenticated as web50m1); Sun, 21 Mar 2010 18:16:08 +0100 (CET) (envelope-from ) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1077) Subject: Re: Test suite blocking release From: Jan Lehnardt In-Reply-To: <4D831DF2-68D2-4BF1-A2A8-77FC3BB0AD05@me.com> Date: Sun, 21 Mar 2010 12:16:08 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <2733FAEB-FB2C-4CFF-A9B6-A60850F447ED@me.com> <700C5F25-8235-4EFA-A3AE-5C330F5FE394@apache.org> <24496B6A-5808-486C-9C95-F0B9E7C55D60@gmail.com> <6AE37542-8E22-4644-95F6-6527B08D4736@apache.org> <66918683-1D1D-4C00-8A31-E11684CEA73D@me.com> <2A53C38E-C053-44D1-BFA3-ECC4C049B99F@apache.org> <4D84D3B7-F601-4C17-94CF-D1053E3A4D8A@me.com> <4D831DF2-68D2-4BF1-A2A8-77FC3BB0AD05@me.com> To: dev@couchdb.apache.org X-Mailer: Apple Mail (2.1077) On 21 Mar 2010, at 12:10, Noah Slater wrote: > What are the CLI tests, if not the etap tests? Are they integrated = into the build system? The CLI tests are the same as the browser tests, just run through our = couchjs binary that has custom HTTP extensions to make the xhr work. At this point I = don't think it is reliable enough to mimic browser behaviour and that we shouldn't use = it for vetting the state of the code. It is very useful when developing new code to not have to switch to and = reload the browser over and over again. Cheers Jan -- >=20 > On 21 Mar 2010, at 17:05, Jan Lehnardt wrote: >=20 >>=20 >> On 21 Mar 2010, at 06:04, Robert Dionne wrote: >>=20 >>> On Mar 21, 2010, at 4:00 AM, Jan Lehnardt wrote: >>>=20 >>>>=20 >>>> On 20 Mar 2010, at 20:06, Paul Davis wrote: >>>>=20 >>>>> On Sat, Mar 20, 2010 at 2:31 PM, Noah Slater = wrote: >>>>>> I think faulty test case should block the release, if I am to = have any >>>>>> future sanity preparing releases. I don't want to delay and = longer, so if >>>>>> you guys are absolutely sure this is a test error and not code = error, then I >>>>>> propose that the test be commented out. Our tests form a contract = between >>>>>> us, internally, and our users. If that contract has a bug, it = should be >>>>>> removed or fixed - or it simply dilutes the importance of = contract. If some >>>>>> one comments out the test, and we agree it is not indicative of = an important >>>>>> bug, I can call the vote within hours. >>>>>>=20 >>>>>=20 >>>>> I'd have to agree on this. =46rom the point of view of a release, = if a >>>>> test reports a failure then it should be made to not report a = failure. >>>>> If that's accomplished by disabling it, then there will be a = commit >>>>> with a message that explains why it was disabled and etc and such = on >>>>> and so forth. >>>>=20 >>>> I'd do that if the test was failing for me :) >>>=20 >>> it's not failing for you when you run changes.js with the CLI ? = Fails for me every time.=20 >>=20 >> I don't consider the CLI tests as part of the official test suite = just yet. >>=20 >> Cheers >> Jan >> -- >>=20 >>>=20 >>> Anyway I poked at this a bit yesterday and am not 100% sure the = issue is in the test. I tried putting a sleep in with no luck. If my = understanding of the JS is correct, CouchDB is supposed to be = synchronous so it's not timing. >>>=20 >>> If someone could comment on the test itself it would be helpful. The = section of the code that fails: >>>=20 >>> // changes get all_docs style with deleted docs >>> var doc =3D {a:1}; >>> db.save(doc); >>> db.deleteDoc(doc); >>> var req =3D CouchDB.request("GET",=20 >>> = "/test_suite_db/_changes?filter=3Dchanges_filter/bop&style=3Dall_docs"); >>> var resp =3D JSON.parse(req.responseText); >>> TEquals(3, resp.results.length, "should return matching rows"); >>>=20 >>>=20 >>> seems odd to me. all_docs as I read the code will return docs with = deletes and conflicts but in this call the filter bop will not apply to = the doc {a:1} so I'm not sure what this delete prior to the call is = about. Anyway I can make it fail in the debugger so perhaps I can find = the root cause. >>>=20 >>>=20 >>>=20 >>>=20 >>>>=20 >>>> Cheers >>>> Jan >>>> -- >>>>=20 >>>=20 >>=20 >=20