httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: Anywhere to go for a summmary of I/O filtering?
Date Thu, 27 Jul 2000 13:19:00 GMT

I've been following (abet, somewhat delayed) the discussion regarding
asynch i/o, non-blocking i/o, and thread level security.  When a couple
people mentioned what OS/390 does, I thought I would explain it a bit more.

The IBM HTTP Server for OS/390 (AKA "Domino Go WebServer", "DGW", "LDGW",
"ICS", ... hey don't blame me, I am a developer!) has a couple modes of
operation. One of those ("standalone") is a multi-threaded, single-process
webserver.  The other mode ("scaleable"), is a multi-threaded,
multi-process mode.  In this mode, there is a single front-end process
which is called a "Queue Manager" which owns all of the network connections
to the clients.  There are 0-n processes, which are called "Queue Servers".
These processes run the user work, i.e. serve files, run cgi's, run
plugin's, etc.

The number of Queue Server's is controlled not by the webserver, but by a
system facility called WLM (Work Load Management).  Each client is
classified by the Queue Manager and placed on a WLM queue.  Associated with
this is an "Application Environment" and "Service Class".  For instance,
the request can be differentiated by the type of resource (cgi vs. file vs.
...), the user, the portion of the url string, etc.   A Queue Server is
started/kept running by WLM to process work associated with an Application
Environment.  WLM also keeps track of how well that type of work is being
performed, and how important that work is.  This importance factor is done
on a system wide basis, and is expressed in human terms.  For instance, I
can have a rule that says that I have certain file get's, e.g. to a portion
of the url tree, and that I want "90% to be completed in <.8 seconds" and
that I give that an importance of 2.   Thus, I can classify the importance
of that work to the system as compared to other work, including other
non-web work.

As a result of this, WLM controls the number of Queue Server's for a given
Application Environment.  (Thus, the 0-n processes.)  Also associated with
each Application Environment is the number of threads per Queue Server.
WLM also controls the underlying system resources, such as cpu and real
memory, etc.
To summarize, the Queue Manager takes each request from the client,
classifies it, places it on a WLM queue, it is picked up by a thread in a
Queue Server (which is started by WLM if required), and the request is

Underlying this is the way asynch i/o works.   As I mentioned above, the
Queue Manager owns the network connection.  The Queue Server thread which
is processing the request doesn't know anything about this, nor the fact
that asynch I/O is being used.  It is synchronous.  But there is special
code in the Queue Server which understands that it is running in scaleable
mode, and it sends a "request for I/O" to the Queue Manager.  This is done
via a IPC message queue and shared memory.  That is, an IPC message is sent
and the data to be written/buffer to be read into is in shared memory.  The
shared memory is allocated by the Queue Manager and is managed as a c-style
heap.  The purpose of the IPC message is essentially a strobe to the Queue

The Queue Manger has a pool of threads which are used to a) process I/o
requests from Queue Servers, b) perform asynch I/O requests, c) get asynch
I/O completion, d) process new incoming accept's.  The Queue Manager has
logic to dynamically balance the number of threads doing each type of work.
(Note that a) and b) are actually combined.)

Getting back to Jeff's? comment, the real issue with threads/processes
being tied to a request isn't so much while the request is being processed,
it is the issue of keepalive.  If the thread needs to be kept in a blocking
read waiting for the next request on a persistent connection,  it is a
problem because you then need to play the game of  how long to tie that
thread up waiting for the client.   In the above design, the Queue
Servers's tell the Queue Manager when it is done processing a request, and
the Queue Server goes on to work on another request.  Thus, a Queue Server
thread isn't tied up waiting on reading the next request on the connection.
The Queue Manager thread does asynch i/o to get the next request, and
therefore doesn't tie up a thread waiting for the client.  (It only costs
us some control block space etc as well as the tcp/ip stack connection

A few other important factors:  I am able to do the shared memory heap pool
management WITHOUT requiring an IPC lock.  This is critical!   Second, the
IPC message queue operations are able to be done w/o IPC lock (in most
cases).  Third, asynch i/o is slower if the data is already available in
the stack.  This is because a normal synchronous read operation would have
been completed without a context switch while the asynch i/o requires one.
Because of this, I use a version of asynch i/o which completes
synchronously if the data is already available.   Fourth, asynch signal
delivery is relatively expensive, thus I try to ensure that there are
threads blocked waiting for the i/o completion.  (The code also works
correctly if there are no threads available and the i/o completes.)


John M. Thompson
Senior Software Engineer
IBM Corporation
Lotus Notes: John Thompson/Poughkeepsie/IBM
VM: thompson at kgnvmc

View raw message