tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig R. McClanahan" <>
Subject Re: Msg formatting problem: trying again
Date Wed, 08 Nov 2000 17:55:34 GMT
Roy Wilson wrote:

> I apologize for the critical tone (but also perhaps ignorance)of my
> [now next to the] last post. I've got the flu or something. Yecch.
Not a problem.

> Here's the data I collected after running new, slightly modified,
> scripts (see attached) for which RequestInfoExample is requested by ab
> 100,000 times on a Celeron 400Mz, 256MB SD100 memory, WD Caviar disk,
> Redhat6.1, with almost nothing else going on:

   C  T secs        X       Connect Processing   Cpu    Cpu     Cpu     T -    Cpu per
              requests/sec    avg    avg (ms)   busy  Ovhead   Ovhead  Ovhead  request
                              (ms)                %   (mins)   (secs)  (secs)    (ms)

   1  2912.78     34.33        0        28       100   0.87     52.2  2860.58    28.6

  10  1495.12     66.88        0        149      100   0.92     55.2  1439.92    14.4

  20  1488.7      67.17        0        297      100   0.98     58.8   1429.9    14.3

  30  1503.03     66.53        0        450      100   1.12     67.2  1435.83    14.4

  40  1591.7      62.83        0        636      100   1.31     78.6   1513.1    15.1
> The earlier results involving the HelloWorldExample servlet were
> affected by variability which I think/hope is averaged out by the large
> number of requests made. The disk, as expected shows only about 1.8
> requests per second, and is not the bottleneck anyway (as expected).
That's right ... once a servlet is loaded it stays in memory, so there
would be little or no I/O in this kind of test scenario.

NOTE:  Of course, that is one of the issues that leads to caution in
interpreting benchmark results as *absolute* indicators of performance
-- this is clearly not going to be representative of the performance of
a servlet that heavily accesses a database, for example.  However, it is
very useful in *relative* performance measurement.  The standard
practices of scientific investigation apply ... tweak one variable at a
time and see what happens.

> My earlier results concerning HelloWorldExample were based on a smaller
> N and were not adjusted to account for CPU overhead. I'll re-run them
> In the earlier results, thread queue locking was observed. Craig noted
> that the number of background threads is set to 75 in the config file.
> With that in mind, I made sure that the ab script ran with 70 or fewer
> concurrent requests. Here's what happened:
> catalina_log*.txt shows "HttpProcessor starting background
> thread[8080][i]" where i runs from 0/1 up to 50.
> catalina.out contains a full thread dump (50 on down), one of which
> involves the message "Waiting to be notified HttpProcessor[8080][43]".
> This appears to be the result of a segmentation violation involving
> HttpProcessor[8080][50] that is also reported. Of course, ab barfs from
> C =3D 50 onward. As I recall, SEGV was also the cause of the thread
> dump=
> when I ran HelloWorldExample. So, the thread dump seems to be a
> repeatable phenomena (on my machine), apparently a function of N & C.
> QUESTION 1: Does segmentation violation occur on other machines? If so,
> is it a problem? If not, why not?
In a pure Java program like Tomcat 4.0-m4, any SEGV violation is always,
100% of the time, a bug in the JVM (or the underlying OS -- and Linux
thread libraries have been known to have issues).  I don't recall which
JVM you are running, but you might want to try at least the 1.2.2 and
1.3.0 releases from Sun (on the same hardware) to compare.  That would
also give some interesting information on how the new HotSpot in 1.3
(and in particular the "server version") affects things.

> QUESTION 2: What do the thruput and average processing times mean? Are
> they "good", "bad", or what?
As discussed above, I would be cautious about treating these numbers as
absolute indicators of expected performance.  What we are after is
identifying places where we can change the code and improve the simple
cases, with the expectation that such improvements will also help in
more realistic cases (even though they will have less impact on the
overall performance, because disk times for database access will tend to

When designing a server platform, most people aim for goals like this:
* Maximum processing times should not have huge
  spikes versus average
* Graceful degredation (ideally linear) as the load
  is increased

Deviations from these goals usually indicate a bottleneck that is being
bumped into, and provides opportunities for tuning.

One thing that will help in this regard is that Sun is making a machine
with some serious hardware (six processors, 3GB of memory, scads of
disk) available for performance tuning.  That should help us shake out
thread-related bugs and performance problems pretty quickly :-).

> QUESTION 3: Since the scripts can be modified to run any servlet, how
> about a "realistic" one as Costin suggested in his ApacheCon
> presentation (unless the examples are "representative").
I agree with this, and would word it slightly differently:  the only
thing a "Hello, World" benchmark predicts is how fast your application
can say "Hello, World".  :-)

If you're after a performance predictor for a real application, you need
a test suite that simulates the actual behavior of the system as closely
as possible -- particularly where the stresses are bumping up against
bottlenecks.  For example, in the test suite you've quoted the system
runs at 100% CPU, so switching to a faster processor (all things else
being equal) would probably make everything run faster -- but adding a
faster disk (or even more memory, if you're not swapping) would not
matter at all.

Benchmarks for performance prediction are an interesting art.

> (Hack) Roy
> Roy Wilson
> E-mail:
>    ----------------------------------------------------------------
> # Draft script by Roy Wilson used on Redhat6.1 system (bash)
> # Assume ONLY HelloWorld servlet being used: better to have a set, but
> what?
> #
> # To promote comparability, run script immediately after a cold boot
> echo Run script only after a cold boot to make results comparable
> echo When done please and post the summary file to Tomcat dev list
> echo Usage:
> echo        sh tcscript servlet-name max-concurrency increment
> echo Example:
> echo        sh tcscript HelloWorldExample 100 10
> rm -f summary
> sh tcscript 1 $1
> load="$3"
> while [ "$load" -le "$2" ]
> do
>         echo "Starting ab with load = " $load
>         sh tcscript $load $1
>         load=$(($load+$3))
> done
>    ----------------------------------------------------------------
> # Roy Wilson 11/2/00
> echo "Starting ab load test with C = " $1 "and servlet = "$2
> rm -f $1$2.sardata
> # Next 2 lines may need adjustment on account of OS differences
> # Start with a fresh temporary system accounting log file
> date >> summary
> cp /var/log/savacct /var/log/tcacct
> /usr/sbin/accton /var/log/tcacct
> # sample CPU and disk usage with 1 second resolution
> (sar -u -b -o $1$2.sardata 1 > /dev/null )&
> sarpid=$!
> # modify path in next line for use of servlet
> /usr/sbin/ab -n 100000 -c $1 http://ALFRED:8080/examples/servlet/$2 >>
> summary
> kill -s TERM ${sarpid}
> # Next line may need modification: Turns off system accounting
> /usr/sbin/accton
> # peel off raw data, keep sar summary line
> sar -f $1$2.sardata 1 | grep Average >> summary
> # collect CPU per minute data from system accounting package
> /usr/sbin/sa -a /var/log/tcacct >> summary
> rm -f $1$2.sardata
> echo "Finished ab load test with C = "$1 "and servlet = "$2
>    ----------------------------------------------------------------
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message