httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Bannert <>
Subject Re: OSDLab projectL Accept() locking on large (6+ CPU) boxes
Date Tue, 24 Jul 2001 18:07:30 GMT
Hi Dirk,

I realize that the OSDlab is [primarily] linux machines, but something
that you might want to look into on Solaris with larger-parallel
machines is the pthread_setconcurrency() call, which AFAIK is not
being called anywhere in httpd or apr.

I quoth from the (Solaris 5.8) man page:

     Unbound threads in a process may or may not be  required  to
     be  simultaneously active. By default, the threads implemen-
     tation ensures that  a  sufficient  number  of  threads  are
     active  so  that  the process can continue to make progress.
     While this conserves system resources, it  may  not  produce
     the most effective level of concurrency.

My interpretation of this is that by default on solaris, pthreads remain
a wholly userspace entity (ie they multiplex one LWP) unless we give a
hint to the OS for how many LWPs we'd like to be able to assign to our
userspace threads.

Ideally, the accept()s are happening randomly across processes in the
threaded MPM, but it is totally possible that a single process is having
to take the burden for a large number of accepts(), while other processes
are sitting idle. Given the above condition with pthread_setconcurrency()
on Solaris, that would mean that a single CPU is now burdened with the
processing requirements of this burdened threaded MPM process, while
other CPUs sit around and yawn.

For more discussion of this, please refer to W. Richard Stevens'
"UNIX Network Programming: Interprocess Communications" Vol 2.

Thanks to Justin for bringing this up last night during his coding binge.


On Tue, Jul 24, 2001 at 10:43:09AM -0700, Dirk-Willem van Gulik wrote:
> Folks,
> Some time ago Sander Temme did some tests to certify Covalent's Apache/SSL
> product as part of the SunTone program.
> Part of that entails running it on anything from a tiny SUN Ultra T1 all
> the way up to 8 way big iron.
> We found that, for purely static content, the default locking mechanisms
> where far from ideal. And in fact that we had a hard time saturating
> machines from small to large with the single configuration across the
> whole range of Sun machines.
> After playing with some locking strategies we now suspect that this
> actually might not be sun specific; but a general multi processor (6 way
> and above) box issue.
> So we've proposed a small project to the the Open Software Developer Lab
> (  to do some careful measuring and tuning attempts across
> a _range_ of 1 CPU .. 16 CPU boxes with different locking mechanisms (on
> Linux in their case). We expect to find that some mechanisms are good for
> few CPU's and some for a lot of CPU's. (And we obviously will experiment
> with various types of loads, client-rtt's, etc, etc to understand what is
> happening).
> In any case; we'll report back here on the list what we find; hopefully
> backed by a patch if we can make apache do better.
> If you have any suggestions, comments; send them either to this list; or
> send them to Sander <> or me - and we'll make sure
> they get summarized.
> If they turn out to be large questions ... well then we'll just need ask
> the OSDLab for some more time or propose a new project.
> But we wanted to keep our first project as simple as possible - and to
> keep its scope small and timeline short.
> Dw.

View raw message