httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zeev Suraski <>
Subject Re: work in progress: mpm-3.tar.gz (fwd)
Date Fri, 18 Jun 1999 17:47:41 GMT
This will probably be rejected on new-httpd, as my subscription address is, whereas my From field is
either or

On Fri, 18 Jun 1999, Dean Gaudet wrote:

> Apache already passes around a request_rec pretty much everywhere... I
> suppose we can implement apache-specific "thread local storage", which we
> save and restore if we ever switch threads... but for first implementation
> I'm really not going to worry about it...
> You could base your storage off the conn_rec * instead of a thread_id... 
> Also, I was planning on building a connection_id which is a densely packed
> small integer, because the scoreboard will need something like this. 

Correct me if I'm wrong, but you can't obtain the conn_rec from anywhere
in the code by calling a simple get_conn_rec_ptr()..?  I guess it'll be
possible to implement such a function (if you keep some mapping between
thread id's and their current corresponding conn_rec's, even though I'm
not sure if you need it for other purposes or whether it would be just
pure added overhead).
Without such a function, you still have to pass the conn_rec pointer
around everywhere, to any function that may possibly need access to a
global per-thread variable, which is exactly what TLS comes to solve...

> The main case I'm considering this for is the handler phase.  In general,
> any request goes through a bunch of protocol stages and reaches the
> handler, and from there it fits into a few small categories:
> 1. copy a file fd back to the client
> 2. copy a pipe/socket fd (from another process) back to the client 
> 3. copy a mmapped region back to the client
> 4. copy a dynamically generated memory region back to the client
> 5. the handler writes stuff to a BUFF, and its sent to the client
> 1, 2, 3, and 4 are very simple cases where if the stuff to be sent doesn't
> fit in the socket's send buffer, and we have to block the thread serving
> the response.  At this point we're potentially consuming an expensive
> resource (a thread, stack, kernel memory for the thread, ...) just to wait
> for the client.
> Instead we can switch to an asynchronous behaviour, all of 1, 2, 3, 4 are
> obvious -- the handler is essentially in a loop which we all know the
> structure of, because the entire object is already generated somewhere.
> The handler at this point sets up a special new record in the conn_rec,
> and will return with a special return code indicating the switch to
> asynchronous behaviour.
> If the MPM supports async stuff, then it will release this thread from
> serving the rest of this request... otherwise there'll be a library to
> handle the "async" stuff synchronously.  This async stuff will be
> completed using select/poll/non-blocking i/o (or other similar variants,
> there are several other faster methods on other platforms).  We've freed
> up the expensive resource:
> - consume less CPU per client
> - handle thousands upon thousands of long haul slow clients, because
>   they're just a conn_rec/request_rec at this point... no kernel stack, no
>   context switching... really, just minimal resource consumption
> - do better on existing benchmarks, but more importantly, do better on
>   real world problems
> At some point in the future the async stuff will finish, and a caller
> supplied "completion" function will be called in another thread.  This
> gets us back out of the async core, and into protocol code, which will
> "resume" the handler.  This way the async core really has no knowledge of
> the protocols involved -- and we can use this technique for any protocol
> (and for variations on 1, 2, 3, 4)...
> 5. is the case for modules which don't want to take advantage of the async
> features.  But we can give them help by turning them into case 4 with more
> features for BUFF... such as buffer up to 50k responses, and do the async
> thing, otherwise do it synchronously. 

Well, I guess it all depends on what kind of stages we're talking about.
If the model is remotely similar to Apache 1.x, then we're not in too much
trouble, since PHP does mostly all of its job in one function call (one
hook).  Several other hooks are the configuration directives which rely on
Apache passing around the configuration struct pointer anyway, so it may
be ok (even though there might be thread-specific resources attached to
this configuration structure, I haven't thought about it thoroughly yet).

> I think you're missing something slightly non-obvious :)  Or I'm still
> missing your point...
> In essence I'm saying that the "thread local storage" is part of the
> conn_rec (or request_rec, whichever is most convenient for you).  All
> entry points into your module include a request_rec structure -- you can
> fetch a void * pointer from your request_data entry; you can store
> whatever you need there. 

Well then, I wasn't missing that, it's just not enough for our purposes :)

One question I raised is whether I can get to that resource from anywhere
in the code even though it wasn't passed on to me through the stack. 

Also, it requires the TLS code to be Apache specific, instead of just
platform specific.  As I said in my previous post, under Win32, for
instance, the same PHP DLL is used for the CGI (which is just a 16KB big
.exe) and the IIS module (which is a 90KB DLL).  Both use the very same
language DLL.  This is more important that one may think, since the fact
that all interfaces work with the same DLL allows you to link other DLLs
against this DLL, and have these other DLLs work with any Win32 interface
of PHP.  For example, if we want to distribute a MySQL extension DLL for
PHP, we won't have to distribute one MySQL extension for CGI, one for IIS,
another for Apache and another for NSAPI - but just one extension DLL,
that's linked against the PHP DLL, that every interface uses.

Which brings me to another consideration - while I haven't read the full
ISAPI spec anywhere (if it even exists) - I think you can rely on all of
the request stages in ISAPI happening within the same thread (Microsoft
certainly uses it in their examples as far as I recall).  If you won't be
able to rely on it in Apache 2.0, it'll pretty much mean that ISAPI will
not be supported..?

> So think of the thread as a resource which happens to execute your code
> for a while, but think of the conn_rec/request_rec as your indication of
> what is going on.
> Apache *could* support "thread local storage", and if this really bothers
> you then I'll encourage you to supply a patch.  We can change the
> requirement this way:
> - MPMs which do not guarantee to use the same thread for all request
> phases must save and restore the thread local storage across such changes
> ... but this is really hard to do portably -- unless we require all
> modules to go through an apache, portable thread local storage API.  Which
> means I'd rather it wait for APR, or rather someone else take care of
> it... 'cause the stuff which I'm working on is busy enough already :)

I'm not exactly sure how we could implement Apache local storage
equivalent, unless we have access to the conn_rec from anywhere in the
code, regardless of function arguments.  If you want TLS support within
Apache you can probably use the TLS code from PHP 4.0 (platform
independant and more powerful than the TLS in Win32), you'd just implement
the function that returns the thread id as a function that returns the
conn_rec pointer or identifier somehow.

Two 'still's remain here though.  It still means that the module would
have to use Apache-specific TLS, which is less than optimal from a point
of view of a module writer that wants the same code to be used on multiple
servers (that would be me in our case:). 

And my general hunch about it is still negative, in the sense that I think
we'd bump into many problems, especially in modules that use 3rd party
libraries that are also thread safe (for example, say you initialize the
MySQL client library in one phase, and then use it in another - there's a
good chance there's thread-specific initialization in that library, and it
won't happen in such a case;  In MySQL's case, I don't think we have a
problem, but I'm almost sure there'd be others in which we would).  And
there's also ISAPI.

How feasible would it be to be able to mark a module as one that wants all
of its request steps to be performed within the same thread?


Zeev Suraski <>
For a PGP public key, finger

View raw message