From derby-dev-return-18132-apmail-db-derby-dev-archive=db.apache.org@db.apache.org Mon Apr 03 22:39:25 2006 Return-Path: Delivered-To: apmail-db-derby-dev-archive@www.apache.org Received: (qmail 96419 invoked from network); 3 Apr 2006 22:39:24 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 3 Apr 2006 22:39:24 -0000 Received: (qmail 48814 invoked by uid 500); 3 Apr 2006 22:39:22 -0000 Delivered-To: apmail-db-derby-dev-archive@db.apache.org Received: (qmail 48730 invoked by uid 500); 3 Apr 2006 22:39:21 -0000 Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: Delivered-To: mailing list derby-dev@db.apache.org Received: (qmail 48698 invoked by uid 99); 3 Apr 2006 22:39:21 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Apr 2006 15:39:21 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=UNPARSEABLE_RELAY X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: local policy) Received: from [192.18.98.34] (HELO brmea-mail-3.sun.com) (192.18.98.34) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Apr 2006 15:39:20 -0700 Received: from phys-mpk-2 ([129.146.11.82]) by brmea-mail-3.sun.com (8.12.10/8.12.9) with ESMTP id k33McuQl013938 for ; Mon, 3 Apr 2006 16:39:00 -0600 (MDT) Received: from conversion-daemon.mpk-mail1.sfbay.sun.com by mpk-mail1.sfbay.sun.com (iPlanet Messaging Server 5.2 HotFix 1.24 (built Dec 19 2003)) id <0IX60040144IYI@mpk-mail1.sfbay.sun.com> (original mail from David.Vancouvering@Sun.COM) for derby-dev@db.apache.org; Mon, 03 Apr 2006 15:38:57 -0700 (PDT) Received: from [129.150.20.211] (vpn-129-150-20-211.SFBay.Sun.COM [129.150.20.211]) by mpk-mail1.sfbay.sun.com (iPlanet Messaging Server 5.2 HotFix 1.24 (built Dec 19 2003)) with ESMTP id <0IX600C7W48Q03@mpk-mail1.sfbay.sun.com> for derby-dev@db.apache.org; Mon, 03 Apr 2006 15:38:50 -0700 (PDT) Date: Mon, 03 Apr 2006 15:38:51 -0700 From: "David W. Van Couvering" Subject: Re: Monitoring/improving code coverage (was Re: code coverage results for trunk - svn revision no 390306) In-reply-to: <4431A283.8010104@sbcglobal.net> To: derby-dev@db.apache.org Message-id: <4431A3FB.3090708@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) References: <836ae9af0603311535x7864b7dct5913e4a7f8ed239d@mail.gmail.com> <442DC13A.2050205@sun.com> <443190B4.2000001@sbcglobal.net> <44319606.20904@sun.com> <4431A283.8010104@sbcglobal.net> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N I think the model we are following with functionality regressions works well. We are not necessarily preventing future checkins unless things are perceived to be Out of Hand by one or more committers. I think the same could be done for code coverage regressions. If a new chunk of code is checked in and the numbers drop way way down for a given module, I think that is cause for concern and a committer could reasonably insist that the feature is not sufficiently tested and back out the change, or at least block any future checkins in that area until enough tests are provided. So I propose having a flag raised when numbers go say 20% below current baseline for a given package, and then it is up to the committers to decide what action is necessary. David Kathey Marsden wrote: > Rick Hillegas wrote: > > >>In previous lives, I've seen code-coverage metrics generated on, say, >>a monthly basis and used as a release barrier. I do not think they are >>appropriate as a barrier to checkin. >> > > I think since we seem to be going for very long release cycles instead > of the release early, release often model, release barriers are not very > effective for maintaining quality on the trunk. Also in open source, > developers come and go so the wait until release model doesn't tend to > work that well in that regard either. If Linux could release a new > kernel twice a day, I tend to think that the trunk should be "ready > for release" at any time for completed functionality and even > incremental functionality could be covered. > > Kathey > >