Mailing-List: contact dev-help@apr.apache.org; run by ezmlm
Precedence: bulk
Message-ID: <1078954058.404f884ad4c2f@webmail.mydomain.com>
Date: Wed, 10 Mar 2004 13:27:38 -0800
From: rbb@rkbloom.net
To: rbb@rkbloom.net
Cc: Joe Orton <joe@manyfish.co.uk>, dev@apr.apache.org
MIME-Version: 1.0
User-Agent: Internet Messaging Program (IMP) 3.1
Subject: Re: cvs commit: apr-site/docs/coverage index.html
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

Quoting rbb@rkbloom.net:

> Quoting Joe Orton <joe@manyfish.co.uk>:
> 
> > On Wed, Mar 10, 2004 at 12:19:25PM -0800, rbb@rkbloom.net wrote:
> > > Also, this data looks slightly better than it is.  While some of the
> files
> > are
> > > in the green block, that doesn't mean that they are fully tested.  Often
> we
> > test
> > > a single error condition which gets us coverage, but that block of code
> > actually
> > > handles 4 different error cases.  If only 1 of 4 is tested, then we are
> > > reporting full coverage when in reality we are about 25% covered.
> > 
> > Yeah, gcov -b gives you the branch coverage, it's usually a better
> > reflection of test coverage.
> 
> I considered using gcov -b, but I couldn't figure out a way to bring the
> data
> down to a single value.  I'll look at this again sometime today to see if I
> can
> generate a similar page using -b.

Oh, and even this wouldn't cover the case I explained above.  As a simple
example, think about apr_file_open.  That can fail for a variety of reasons that
all exercise the same branch of code.  For example, open() can return EEXIST or
EACCES.  Both of these results from open() exercise the same branch of code, so
that branch will be reported as having been fully exercised when in reality it
hasn't been, because we only tested one of the possible errors from open().  We
use this design where an error in the native function is just returned from the
APR function throughout APR.  

Because of the way APR works, where we try to duplicate common errors across
platforms, this is an important case to consider.  Gcov will never detect this
for us, and it just means that our coverage will always be over reported.  This
isn't bad, and we shouldn't expect gcov to handle this case for us, but we do
need to be aware of it.

Ryan