httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Eissing <stefan.eiss...@greenbytes.de>
Subject Re: buckets across threads - question
Date Fri, 20 Mar 2015 09:12:08 GMT
Graham,

thanks for the explanations and the hint to mod_proxy. I am now able to use pools/subpools
and bucket_alloctors in a better way which improves overall performance.

Just for the record: the important thing to learn for me was that there is another layer underneath
apr_pool_t called apr_allocator_t which is not  thread-safe by default - but can be made so.
With my own allocator per real connection and a mutex protecting it, I can pool and subpool
to my heart's content across threads. In my measurements, the disadvantage of a mutex'ed allocator
is much overcompensated by proper pool/memory reuse, e.g. it is faster.

Cheers,

  Stefan


> Am 17.03.2015 um 23:52 schrieb Graham Leggett <minfrin@sharp.fm>:
> 
> On 17 Mar 2015, at 15:59, Stefan Eissing <stefan.eissing@greenbytes.de> wrote:
> 
>> 
>> Hi,
>> 
>> I have a question regarding my http2 module implementation, where I seem to fail
to wrap my head around the limitations of pools and bucket brigades in a multi-threaded environment.
As I understand it, httpd has - so far - the underlying assumption:
>> 
>> 1 connection -> 1 request -> 1 thread
>> 
>> at a single point in time. This used to be quite static, as in mpm_worker, and was
made more dynamic with mpm_event, which may switch the thread around. However even though
entities might be exchanged, it is always 1-1-1 at a single point in time.
>> 
>> Now, as most of you are aware, with the HTTP/2 processing model, this holds no longer
true. The relation there is 1-m-m* and, worse, the connection is multiplexing requests, so
there are many active at the same time and their data will be interleaved. The request/thread
coupling can stay as it is, but the connection relation is different.
>> 
>> I today have a working implementation and it has major drawbacks (among all the bugs
I need to find):
>> 
>> 1. it mimicks httpd objects that are not really there, e.g. pseudo-connections, so
that there seems to be a 1-1 with connections and requests again (just as mod_spdy does).
Therefore it needs to fake HTTP/1.1 requests, so that core filter can parse them, etc.
>> 
>> 2. beautiful apr infrastructure like bucket brigades are not used when data needs
to be passed between threads.
>> 
> 
>> I have made an attempt today to make bucket brigades work for passing data between
threads, but things keep crashing on me when load grows. Which is all my fault for not understanding
these things fully. So, am I on a fool's errand? Can I have brigade A in thread 1 and brigade
B in thread 2 and pass buckets between them without excessive copying? I think I am doing
it wrong and the cleanups of the allocators run in the wrong thread in the end which is not
healthy.
> 
> One place where you can get a concrete example of data in one pool lifetime being transferred
across to another is in mod_proxy - the backend connection comes out of a pool, and may live
longer or shorter than the request going out the front.
> 
> Getting this right is tricky, you need to be particular about the pool lifetimes.
> 
> Pools create a hierarchy, that is the key. If you have requests subordinate to a connection,
ie when the connection dies the requests die, let the request be a subpool of the connection.
> 
> You may have many independent requests linked to the same connection, just make sure
the requests don't leak into each other.
> 
> Regards,
> Graham
> --

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782




Mime
View raw message