gump-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Berin Loritsch <blorit...@apache.org>
Subject Re: FOG Factor
Date Thu, 05 Jun 2003 13:26:33 GMT
Adam Jack wrote:
> For some reason I got to thinking about the "Friend of Gump" rating again
> today. (Probably 'cos I am curious to know how centipede would build on full
> gump, but know it hasn't built in a week or many more 'cos of dependent
> failures). As product get higher up the stack they build less, and that
> limits gumps effectiveness at keeping community unity.  As such, I think FOG
> is a very important statistic.
> 
> I am sure there are some interesting statistics (total # of sub-projects a
> project depends upon, total # of project dependent upon this) I wonder if
> there is a value in the following simple approach.
> 
> Gump could store three counters for each project (1) successful builds [S]
> (2) unsuccessful builds [U] (2) missing dependency [D]. I wonder if the "FOG
> factor" ought be:
> 
> 	[S] / [U] + [D].
> 
> I thought about weighting [U] or [D], so as not to penalise a project for
> it's dependencies, but the end result is the same. Build on quicksand and
> you'll sink, however stable your are.

One of the things about GUMP is that it is an _all_ or _nothing_ thing.
Either ALL of your dependencies build or your project fails--even if the
dependency is optional.  If your build can survive one or two different
dependencies not being present, should you _really_ be penalized for it?

I mean if we are going to do an FOG type of thing, we really need to
take into consideration the possibility of *optional* dependencies.

A modified version of the algorithm above accounting for optional
dependencies might look like this:

( S + O/Ot ) / ( U + Rt/R )

S  = Successful build
O  = Successful Optional dependency
Ot = Total number of Optional dependencies
U  = Unseccessful build
R  = Successful Required dependency
Rt = Total number of Required dependencies

Which means the numerator is augmented for every successful optional
dependency and the denominator is decremented for every unsuccessful
required dependency.

You would find that project like Centipede that are very modular
would fare much better, and it would encourage that modular way
of development.

For something like Foo Project, we would have:

Optional Dependency: DBPool, XMLPersistance
Required Dependency: XML-APIs, Bar Project

If all works properly:

(1 + 2/2) / (0 + 2/2) = (1 + 1) / (1) = 2/1 = 2

If an optional dep fails the second time:

(2 + 1/2) / (0 + 2/2) = (2.5) / (1) = 2.5

If a required dep fails the third time:

(2 + 2/2) / (1 + 2/1) = (2.5) / (1 + 2) = 2.5/3 = 0.83

And if two optionals fail the fourth time:

(3 + 0/2) / (1 + 2/2) = (3) / (1 + 1) = 3/2 = 1.5

As you can see, failures hurt, but there is a difference
between a true dependency and an optional dependency.

The FOG value of 1.5 after four builds versus the FOG value if
everything was successful in every build would be 5.  As you can
see there is a penalty.  If the optional dependencies failed every
time though the result would be 4.

There are some problems with this approach, so it is not perfect.
For example: 0/0 would have to be interpreted as 1 as opposed to
NaN.  That would make the deps to evaluate to 1 if there are no
dependencies and avoids the DivideByZeroException if there are no
failures.

For example something like Basic Project that has no dependencies
whatsoever would be evaluated strictly by its pass/fail rate.
If it passes four times, its FOG value would be 5 (i.e. num Passes
+ [ 0/0 == 1 ] ).  If it failed 2 out of four times it would be
1.  If it failed one out of four times it would be 2.



Mime
View raw message