Paul J. Reder wrote:
>
>
> Brian Pane wrote:
>
>> I've been thinking about strategies for building a
>> multiple-connection-per-thread MPM for 2.0. It's
>> conceptually easy to do this:
>>
>> * Start with worker.
>>
>> * Keep the model of one worker thread per request,
>> so that blocking or CPU-intensive modules don't
>> need to be rewritten as state machines.
>>
>> * In the core output filter, instead of doing
>> actual socket writes, hand off the output
>> brigades to a "writer thread."
>
>
>
> During a discusion today, the idea came up to have the
> code check if it could be written directly instead of
> always passing it to the writer. If the whole response
> is present and can be successfully written, why not save
> the overhead. If the write fails, or the response is too
> complex, then pass it over to the writer.
>
>
>>
>> * As soon as the worker thread has sent an EOS
>> to the writer thread, let the worker thread
>> move on to the next request.
>
>
>
> I have a small concern here. Right now the writes are
> providing the throttle that keeps the system from generating
> so much queued output that we burn system resources. If
> we allow workers to generate responses without a throttle,
> it seems possible that the writer's queue will grow to the
> point that the system starts running out of resources.
>
maybe if we something like the queue (apr-util/misc/apr_queue.c)
to submit the write requests, we could limit the number of outstanding
writes to X, with the threads sleeping when the queue gets full.
I'm actually working on a dynamicly growing thread pools which would
read the queue, and adjust the number of threads based on the size of
the queue (eventually I want to adjust the number of threads based on
the response time)
If anyone is interested (currently buggy) code, I'll put it up on
webperf somewhere
> Only testing will show for sure, and maybe in the real world
> it would only happen for brief periods of heavy load, but it
> seems like we need some sort of writer queue thresholding
> with pushback to control worker throughput.
>
> Of course, if we do add a throttle for the workers, then how
> does this really improve things? The writer was the throttle
> before and it would be again. We've added an extra queue so
> there will be a period of increased worker output until the
> queue threshold is met but, once the queue is filled, we revert
> to the writer being the throttle. The workers cannot finish
> their current response until the writer has finished writing
> a queued response and freed up a queue slot.
>
>>
>> * In the writer thread, use a big event loop
>> (with /dev/poll or RT signals or kqueue, depending
>> on platform) to do nonblocking writes for all
>> open connections.
>>
>> This would allow us to use a much smaller number of
>> worker threads for the same amount amount of traffic
>> (at least for typical workloads in which the network
>> write time constitutes the majority of each requests's
>> duration).
>>
>> The problem, though, is that passing brigades between
>> threads is unsafe:
>>
>> * The bucket allocator alloc/free code isn't
>> thread-safe, so bad things will happen if the
>> writer thread tries to free a bucket (that's
>> just been written to the client) at the same
>> time that a worker thread is allocating a new
>> bucket for a subsequent request on the same
>> connection.
>>
>> * If we delete the request pool when the worker
>> thread finishes its work on the request, the
>> pool cleanup will close the underlying objects
>> for the request's file/pipe/mmap/etc buckets.
>> When the writer thread tries to output these
>> buckets, the writes will fail.
>>
>> There are other ways to structure an async MPM, but
>> in almost all cases we'll face the same problem:
>> buckets that get created by one thread must be
>> delivered and then freed by a different thread, and
>> the current memory management design can't handle
>> that.
>>
>> The cleanest solution I've thought of so far is:
>>
>> * Modify the bucket allocator code to allow
>> thread-safe alloc/free of buckets. For the
>> common cases, it should be possible to do
>> this without mutexes by using apr_atomic_cas()
>> based spin loops. (There will be at most two
>> threads contending for the same allocator--
>> one worker thread and the writer thread--so
>> the amount of spinning should be minimal.)
>>
>> * Don't delete the request pool at the end of
>> a request. Instead, delay its deletion until
>> the last bucket from that request is sent.
>> One way to do this is to create a new metadata
>> bucket type that stores the pointer to the
>> request pool. The worker thread can append
>> this metadata bucket to the output brigade,
>> right before the EOS. The writer thread then
>> reads the metadata bucket and deletes (or
>> clears and recycles) the referenced pool after
>> sending the response. This would mean, however,
>> that the request pool couldn't be a subpool of
>> the connection pool. The writer thread would have
>> to be careful to clean up the request pool(s)
>> upon connection abort.
>>
>> I'm eager to hear comments from others who have looked
>> at the async design issues.
>>
>> Thanks,
>> Brian
>>
>>
>>
>
>
|