httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul J. Reder" <>
Subject Re: Bucket management strategies for async MPMs?
Date Wed, 04 Sep 2002 02:13:30 GMT

Brian Pane wrote:

> I've been thinking about strategies for building a
> multiple-connection-per-thread MPM for 2.0.  It's
> conceptually easy to do this:
>  * Start with worker.
>  * Keep the model of one worker thread per request,
>    so that blocking or CPU-intensive modules don't
>    need to be rewritten as state machines.
>  * In the core output filter, instead of doing
>    actual socket writes, hand off the output
>    brigades to a "writer thread."

During a discusion today, the idea came up to have the
code check if it could be written directly instead of
always passing it to the writer. If the whole response
is present and can be successfully written, why not save
the overhead. If the write fails, or the response is too
complex, then pass it over to the writer.

>  * As soon as the worker thread has sent an EOS
>    to the writer thread, let the worker thread
>    move on to the next request.

I have a small concern here. Right now the writes are
providing the throttle that keeps the system from generating
so much queued output that we burn system resources. If
we allow workers to generate responses without a throttle,
it seems possible that the writer's queue will grow to the
point that the system starts running out of resources.

Only testing will show for sure, and maybe in the real world
it would only happen for brief periods of heavy load, but it
seems like we need some sort of writer queue thresholding
with pushback to control worker throughput.

Of course, if we do add a throttle for the workers, then how
does this really improve things? The writer was the throttle
before and it would be again. We've added an extra queue so
there will be a period of increased worker output until the
queue threshold is met but, once the queue is filled, we revert
to the writer being the throttle. The workers cannot finish
their current response until the writer has finished writing
a queued response and freed up a queue slot.

>  * In the writer thread, use a big event loop
>    (with /dev/poll or RT signals or kqueue, depending
>    on platform) to do nonblocking writes for all
>    open connections.
> This would allow us to use a much smaller number of
> worker threads for the same amount amount of traffic
> (at least for typical workloads in which the network
> write time constitutes the majority of each requests's
> duration).
> The problem, though, is that passing brigades between
> threads is unsafe:
>  * The bucket allocator alloc/free code isn't
>    thread-safe, so bad things will happen if the
>    writer thread tries to free a bucket (that's
>    just been written to the client) at the same
>    time that a worker thread is allocating a new
>    bucket for a subsequent request on the same
>    connection.
>  * If we delete the request pool when the worker
>    thread finishes its work on the request, the
>    pool cleanup will close the underlying objects
>    for the request's file/pipe/mmap/etc buckets.
>    When the writer thread tries to output these
>    buckets, the writes will fail.
> There are other ways to structure an async MPM, but
> in almost all cases we'll face the same problem:
> buckets that get created by one thread must be
> delivered and then freed by a different thread, and
> the current memory management design can't handle
> that.
> The cleanest solution I've thought of so far is:
>  * Modify the bucket allocator code to allow
>    thread-safe alloc/free of buckets.  For the
>    common cases, it should be possible to do
>    this without mutexes by using apr_atomic_cas()
>    based spin loops.  (There will be at most two
>    threads contending for the same allocator--
>    one worker thread and the writer thread--so
>    the amount of spinning should be minimal.)
>  * Don't delete the request pool at the end of
>    a request.  Instead, delay its deletion until
>    the last bucket from that request is sent.
>    One way to do this is to create a new metadata
>    bucket type that stores the pointer to the
>    request pool.  The worker thread can append
>    this metadata bucket to the output brigade,
>    right before the EOS.  The writer thread then
>    reads the metadata bucket and deletes (or
>    clears and recycles) the referenced pool after
>    sending the response.  This would mean, however,
>    that the request pool couldn't be a subpool of
>    the connection pool.  The writer thread would have
>    to be careful to clean up the request pool(s)
>    upon connection abort.
> I'm eager to hear comments from others who have looked
> at the async design issues.
> Thanks,
> Brian

Paul J. Reder
"The strength of the Constitution lies entirely in the determination of each
citizen to defend it.  Only if every single citizen feels duty bound to do
his share in this defense are the constitutional rights secure."
-- Albert Einstein

View raw message