Mailing-List: contact new-httpd-help@apache.org; run by ezmlm
Precedence: bulk
Reply-To: new-httpd@apache.org
Date: Tue, 24 Jul 2001 11:45:32 -0700
From: Aaron Bannert <aaron@ebuilt.com>
To: new-httpd@apache.org
Subject: Re: OSDLab projectL Accept() locking on large (6+ CPU) boxes
Message-ID: <20010724114532.M28176@ebuilt.com>
References: 
 <Pine.OSX.4.33.0107241039310.3569-100000@titatovenaar.sfo.covalent.net>
 <20010724110730.L28176@ebuilt.com> <3B5DBF39.2000609@pacbell.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5i
In-Reply-To: <3B5DBF39.2000609@pacbell.net>;
 from bpane@pacbell.net on Tue, Jul 24, 2001 at 11:32:25AM -0700

On Tue, Jul 24, 2001 at 11:32:25AM -0700, Brian Pane wrote:
> Aaron Bannert wrote:
> 
> >Hi Dirk,
> >
> >I realize that the OSDlab is [primarily] linux machines, but something
> >that you might want to look into on Solaris with larger-parallel
> >machines is the pthread_setconcurrency() call, which AFAIK is not
> >being called anywhere in httpd or apr.
> >
> >I quoth from the (Solaris 5.8) man page:
> >
> >     Unbound threads in a process may or may not be  required  to
> >     be  simultaneously active. By default, the threads implemen-
> >     tation ensures that  a  sufficient  number  of  threads  are
> >     active  so  that  the process can continue to make progress.
> >     While this conserves system resources, it  may  not  produce
> >     the most effective level of concurrency.
> >
> >
> >My interpretation of this is that by default on solaris, pthreads remain
> >a wholly userspace entity (ie they multiplex one LWP) unless we give a
> >hint to the OS for how many LWPs we'd like to be able to assign to our
> >userspace threads.
> >
> Are you able to observe this effect experimentally too?  E.g., if
> you run the threaded MPM on Solaris, does it use just one LWP per
> process?

I have observed this under conditions where the scheduling was multiplexed
with a pthread_mutex (and only on a uniprocessor machine), but AFAIK
we don't always use pthread_mutex to multiplex access to the accept()
call (ie we don't have to use it when there is no "thundering herd"
problem).  ISTR apache-1.3 used fcntl() as the global-locking mechanism
that multiplexed calls around a "thundering herd" accept() call. All
of these issues, in my mind, could alter the way the OS decides to
schedule blocked LWPs that receive an event.

I will try to do some tests later today on our 2-way solaris machines
to see if the CPU usage seems to favor one process over another, and
then I'll observe the usage under the same load with pthread_setconcurrency()
stuck in there somewhere.

The hard part about this (at least for me) is the multitude of variables
that go in to deciding the implementation of the accept() mutex. Is there
a way I can find out (at runtime) what implementation is being used?

-aaron