httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Reid" <dr...@jetnet.co.uk>
Subject Re: Can AB be compared ?
Date Wed, 01 May 2002 18:59:23 GMT
FWIW, is you don't have writev and are not using ssl, ab in the 1.3 tree is
broken and won't even compile at present :(

We really should fix it - but maybe the person who broke it could do that as
I'm sure they know what they were intending with the changes.

david

----- Original Message -----
From: <dirkx@covalent.net>
To: <dev@httpd.apache.org>
Sent: Wednesday, May 01, 2002 4:05 PM
Subject: Can AB be compared ?


>
> Just to have some fun - Below is the result of running a build of AB
> against the same apache 1.3.0 (stock)
>
> It is a simple loop - checkout against a tag; cd apache-1.3/src && cp
> Configuration.tmp && Configuration && ./Configure && make &&
cd support &&
> make) and then run 20 times ./ab -c 30 -n 10000 and take the last 25 runs.
>
> By the way - not a -single- tag failed to build on FreeBSD 5-Current and
> not a single tag needed a differnet config ! A record hard to beat by
> commercial software !
>
> For each is displayed to total avaerage time/connection; the 98%
> confidence interval (all in mSec) and the averegate for that version
> of AB.
>
> Note that typically (with eceptiosn early on) each version of AB produces
> run within the confidence interval - and that successive versions their
> confidence intertvals do generally not overlap (except early in the
> development) in the more recent series.
>
> So though this shows that comparing versions of AB from different versions
> is generally waranted (same mean, same SD) - for recent versions of
> 1.3 - and that comparisions between version no's is not a good idea.
>
> BUT at the same time - that has not always been the case - and that is is
> kind of a miracle (or a tribute to release management/fact that AB hardly
> ever changed) that comparison is possible. Also if you include 1.3c and
> 1.3a then you can conclude that these ranges match - thus from these
> measurements you would conclude that the code did not change substantially
> (it may of course do for keep alive or something).
>
> Dw.
>
>         Tag checout     #define in ab   mean/SD aver   CI
> Tag:    APACHE_1_3_24   AB:     1.3d    66 +-3  66.5 +-3
> Tag:    APACHE_1_3_23   AB:     1.3d    67 +-3
> Tag:    APACHE_1_3_22   AB:     1.3d    67 +-3
> Tag:    APACHE_1_3_21   AB:     1.3d    65 +-3
>
> Tag:    APACHE_1_3_20   AB:     1.3c    63 +-2  61.8 +-2
> Tag:    APACHE_1_3_19   AB:     1.3c    60 +-2
> Tag:    APACHE_1_3_18   AB:     1.3c    61 +-2
> Tag:    APACHE_1_3_17   AB:     1.3c    61 +-2
> Tag:    APACHE_1_3_16   AB:     1.3c    63 +-2
> Tag:    APACHE_1_3_15   AB:     1.3c    62 +-2
> Tag:    APACHE_1_3_14   AB:     1.3c    63 +-2
> Tag:    APACHE_1_3_13   AB:     1.3c    62 +-2
> Tag:    APACHE_1_3_12   AB:     1.3c    62 +-2
> Tag:    APACHE_1_3_11   AB:     1.3c    60 +-2
> Tag:    APACHE_1_3_10   AB:     1.3c    63 +-2
>
> Tag:    APACHE_1_3_9    AB:     1.3a    60 +-1  60.4 +-1
> Tag:    APACHE_1_3_8    AB:     1.3a    61 +-1
> Tag:    APACHE_1_3_7    AB:     1.3a    61 +-1
>
> Tag:    APACHE_1_3_6    AB:     1.3     64 +-2  65.2 +-2
> Tag:    APACHE_1_3_5    AB:     1.3     66 +-3
>
> Tag:    APACHE_1_3_4    AB:     1.2     61 +-3  62.9 +-3
> Tag:    APACHE_1_3_3    AB:     1.2     61 +-4
> Tag:    APACHE_1_3_2    AB:     1.2     64 +-3
>
> Tag:    APACHE_1_3_1    AB:     1.1     60 +-9  65.8 +-10
> Tag:    APACHE_1_3_0    AB:     1.1     67 +-6
> Tag:    APACHE_1_3b7    AB:     1.1     64 +-9
> Tag:    APACHE_1_3b6    AB:     1.1     79 +-8
>
> Dw
> --
> Dirk-Willem van Gulik
>
>
>


Mime
View raw message