apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r..@rkbloom.net
Subject Re: cvs commit: apr-site/docs/coverage index.html
Date Wed, 10 Mar 2004 21:27:38 GMT
Quoting rbb@rkbloom.net:

> Quoting Joe Orton <joe@manyfish.co.uk>:
> 
> > On Wed, Mar 10, 2004 at 12:19:25PM -0800, rbb@rkbloom.net wrote:
> > > Also, this data looks slightly better than it is.  While some of the
> files
> > are
> > > in the green block, that doesn't mean that they are fully tested.  Often
> we
> > test
> > > a single error condition which gets us coverage, but that block of code
> > actually
> > > handles 4 different error cases.  If only 1 of 4 is tested, then we are
> > > reporting full coverage when in reality we are about 25% covered.
> > 
> > Yeah, gcov -b gives you the branch coverage, it's usually a better
> > reflection of test coverage.
> 
> I considered using gcov -b, but I couldn't figure out a way to bring the
> data
> down to a single value.  I'll look at this again sometime today to see if I
> can
> generate a similar page using -b.

Oh, and even this wouldn't cover the case I explained above.  As a simple
example, think about apr_file_open.  That can fail for a variety of reasons that
all exercise the same branch of code.  For example, open() can return EEXIST or
EACCES.  Both of these results from open() exercise the same branch of code, so
that branch will be reported as having been fully exercised when in reality it
hasn't been, because we only tested one of the possible errors from open().  We
use this design where an error in the native function is just returned from the
APR function throughout APR.  

Because of the way APR works, where we try to duplicate common errors across
platforms, this is an important case to consider.  Gcov will never detect this
for us, and it just means that our coverage will always be over reported.  This
isn't bad, and we shouldn't expect gcov to handle this case for us, but we do
need to be aware of it.

Ryan

Mime
View raw message