httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael H. Voase" <>
Subject Re: More on early multiplexing.
Date Mon, 29 Mar 1999 16:56:09 GMT

	Looking back at me former message , I can see
I should clarify this algorithm a bit ...
( Note - THS in this passage stands for 'thundering heard syndrome' . I
am tired of writing this out fully .. ;-P )

Ryan Bloom wrote:
> On Sun, 21 Mar 1999, Dean Gaudet wrote:
> >
> >
> > On Thu, 11 Mar 1999, Michael H. Voase wrote:
> >
> > > blocked threads --> (server socket queue )  <---------------------
> > > ^                               |                                 |
> > > |                               V                                 |
> > > |                        thread blocked on accept on socket X     |
> > > |                               |                                 |
> > > |                               V                                 |
> > > |                        Unblocked by a request the thread -------
> > > |                        enqueues server socket back onto queue
> > > |                         wanders off with the client socket to
> > > |                        service request
> > > |                               |
> > > |                               V
> > > L_______________________ Thread completes the request
> > >                          and rejoins queue for the next
> > >                          server socket
> >
The last bit just above referring to the thread rejoining
a 'queue' just simply means blocking on the mutex around the 
the server socket queue . My apologies if this has caused
confusion .

> > I think the main problem with this is that it has two synchronization
> > points -- you have to block on the queue on the way into and the way out
> > of accept.
> It has taken me a while to be able to express the concerns I have with
> this algorithm, but I'll give it a try.
> There are two cases that should be considered.
> 1)  An OS with a thundering herd problem, in which case the queue would
> have to go in shared memory, or we need another synchronization point.
> With thundering herd, we want only one thread in the entire server
> listening on the port.  By putting the queue in shared mem, we get that.
> If we don't use shared mem, the other synchronization point is so that
> each thread that gets a socket descriptor from it's process's queue has to
> check that the port isn't being listened to by another process.

To clarify what I was suggesting , yes the server socket queue should
( (w|c)ould ? ) be shared only in the threads serving the the port(s)
in the queue . Only one instance of each port being serviced should
be present in the queue loop . If one builds a server such that there 
is one queue per process ( with of course multiple threads in that 
process ) one could allocate what sockets are to be serviced by that 
process . The idea can then be expanded to accomidate different builds 
of server . That is , if one wanted a child process built with ,say , 
mod_perl servicing another port , then a distinct build could be used 
in its own process with its own queue and have the number of threads
allocated to how much resource you could afford to feed it with ;-)

	One of the core advantages of this queue is that it has a
very similar code structure to the 'all on accept' method that Apache 
currently uses . The idea being that it should be reasonably possible
to implement the queue along side existing apache code . By allowing 
the user to be able to select type of accept method apache is built
with , the code could achieve some flexibility ( equivalent of 1.3.x
style in the same code base ) . I hope that this satisfies Ryans two 
conditions that he has pointed out . 	
	Secondly , with THS  the server socket queue would have the
same process control qualities that the acceptor loop method already
has . Basically , the acceptor loop uses cond_signal and cond_wait
functions to signal the worker threads . 

> 2)  An OS without a thundering herd problem, in which case, we want as
> many threads as possible listening to a port.  This algorithm gives us
> that ability, but each process would need it's own queue.
> These two cases are at odds with each other, and I am having a hard time
> resolving it.

	 Yes they are at odds with each other . Considering that 
there are OS's out there dont suffer from THS and thus can use the 
classic 'all-on-accept' method quite successfully , then there is no
real need to have a server socket queue or an acceptor loop . Let me
see ... mmm I remember a message that went something like ...


Another solution is to have all threads block on the accept.  Then, you
take the chance of the OS having a problem with a thundering herd.  If
threads block on the accept, and a request comes in, many OS's will wake
up all of the threads, and then put them back to sleep after the first
thread has gotten the request.  This causes many many context switches.
BAD :(  There is also the potential, out of pure dumb luck, that two
threads will get through to the accept loop at once (IBM's last web
had this problem when they used this algorithm), this will cause bad
things to happen. 


	the server socket queue would have the same essential characteristics
that allows acceptor loop to perform well on THS crippled servers .
it retains a more traditional code structure . 

> The two synchronization points doesn't concern me.  The current fdqueue
> implementation has two points of synchronization(1 in and 1 out), and it
> is very fast.  The testing we have done seems to indicate that one process
> handles all of the load until it is pegged, and then another process helps
> out.

Hrrm ... interesting .

> Ryan
> _______________________________________________________________________
> Ryan Bloom    
> 4205 S Miami Blvd
> RTP, NC 27709           It's a beautiful sight to see good dancers
>                         doing simple steps.  It's a painful sight to
>                         see beginners doing complicated patterns.

	Lastly as a note to this concept , I have doubts as to whether
this algorithm ( or Ryan's for that matter ) could beat the classic 
accept method that is tried and tested on machines that don't suffer 
from THS . The server socket queue does have a small advantage that it 
does not need select() to detect an incoming request , but that would 
probably be offset by the added overhead of the queue .

As Dean points out earlier there is two sync points in server socket
queue , 
classic Apache only has one - the accept mutex 

	One interesting ramification in this algorithms
favour is that it does give a logical 'separation' point for
servers of mixed builds .

	With a server socket queue layout one could allocate 
each build set to a process with its own queue , declare what
port(s) each process would be listening to and allocate the 
number of threads to suit . This could be made optional with
the all-on-accept method ready to use instead , with each process
being allocated its build type then its sockets . Since 
the classic method could accept on multiple sockets , it would
be trivial to implement it as the default with each process having its
own mutex and FDSET . Further more , this conceptualisation can 
contribute back to the core code base by formalising a definition
for mixed build servers with individual mutexes . Being able to 
configure a mix of builds in one config file could be of some 
benefit to web masters . no ?

	Just a couple of thoughts anyways

Cheers Mik Voase.

 /~\     /~\            CASTLE INDUSTRIES PTY. LTD.
 | |_____| |            Incorporated 1969. in N.S.W., Australia
 |         |            Phone +612 6562 1345 Fax +612 6567 1449
 |   /~\   |            Web
 |   [ ]   |            Michael H. Voase.  Director.
~~~~~~~~~~~~~~          The only difference between a car salesman and
a computer salesman is that the car salesman knows he's lying .

View raw message