gump-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [jira] Commented: (GUMP-21) FOG factor is misleading and requires improvement
Date Fri, 12 Mar 2004 23:45:17 GMT
The following comment has been added to this issue:

     Author: Adam Jack
    Created: Fri, 12 Mar 2004 3:43 PM
I'd love to fix this, I agree with much of the objection, but I'd need suggestions. I suspect
there is no such thing as a single 'FOG Factor' (to suit all purposes) but maybe it is a quick

I think we need to factor in 'depth' (how much re-use a project leverages and cut it some
slack for the good intentions.) Note: FOG Factor ought factor in failures due to an unstable
stack, so we shouldn't ignore pre-req failures, maybe just weight them. Hmm, maybe we (somehow)
factor in the 'depth' (max tree of dependencies) of the 'cause' that took it down.

Any ideas welcomed...
View this comment:

View the issue:

Here is an overview of the issue:
        Key: GUMP-21
    Summary: FOG factor is misleading and requires improvement
       Type: Improvement

     Status: Unassigned
   Priority: Major

    Project: Gump

   Reporter: Daniel Savarese

    Created: Thu, 4 Mar 2004 5:21 PM
    Updated: Fri, 12 Mar 2004 3:43 PM

FOG is defined right now as:
def getFOGFactor(self):
    return round((float(self.successes) / (float(self.failures) + float(self.prereqs))), 2)

This is not helpful.  Last month, jakart-oro
failed to build by coincidence around the time of a Gump
run and the problem was fixed before the next Gump run,
yet its FOG factor dropped in half.  The responsiveness
to resolving build and integration problems is not taken
into account.  It is likely that it is impossible for FOG
to mean anything because the factors that determine
a project's "health" (to borrow the term used by Adam Jack
in an email to the commons-dev and gump lists) are largely
orthogonal.  FOG should probably be a vector and if you really
wanted a single number you could use the length of that vector.

One of the major problems with the current factor is that
a project's FOG factor continues to increase when no changes
have been made to the repository.  It's not meaningful to
count only successful builds towards improving the factor
because no changes may have been made to the code base.
Ideally, you count only builds that cause no failures after
a change to the code base.  FOG doesn't factor in how quickly
(or slowly) projects recover from build failures, and how often
changes to a code base cause failures (either in itself or in
other projects).  Right now, a project with few code changes
(say once a month) that causes a failure 50% (1 every 2 months)
of the time the code is changed with a mean time of failure
correction of 1 week is deemed more reliable (higher FOG factor)
than a project that makes frequent changes (say every day) to
its code base and causes a failure 10% of the time (1 every 10
days) with a mean time of failure correction of 1 day.  That's
because FOG is based almost entirely on consecutive successful
Gump builds, which is not useful.

Rather than propose a new meaningless way of computing FOG, I
suggest FOG stop being used temporarily.  It would be more
useful to break out and report all of the independently useful
metrics and through experience over time determine how each
impacts the overall notion of FOG and eventually derive a
way of condensing the information (perhaps a FOG factor,
perhaps a vector) in a useful manner.  Right now the state
of affairs is that someone decided FOG was a cool thing, so
it's being reported and people see this number and mistakenly
think a project with half the FOG factor of another is
less dependable and therefore avoid using it.  Alternatively
an explanation of what FOG is intended to compute and how
it is computed should be placed on Gump reports that list
FOG so that consumers of the information will not be misled.

This message is automatically generated by JIRA.

If you think it was sent incorrectly contact one of the administrators:

If you want more information on JIRA, or have a bug to report see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message