httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Eissing <>
Subject Re: buckets and connections (long post)
Date Thu, 22 Oct 2015 16:03:26 GMT

> Am 21.10.2015 um 16:48 schrieb Graham Leggett <>:
> On 21 Oct 2015, at 4:18 PM, Stefan Eissing <> wrote:
>> 6. pool buckets are very tricky to optimize, as pool creation/destroy is not thread-safe
in general
>>  and it depends on how the parent pools and their allocators are set up. 
>>  Early hopes get easily crushed under load.
> As soon as I see “buckets aren’t thread safe” I read it as “buckets are being
misused” or “pool lifetimes are being mixed up".
> Buckets arise from allocators, and you must never try add a bucket from one allocator
into a brigade sourced from another allocator. For example, if you have a bucket allocated
from the slave connection, you need to copy it into a different bucket allocated from the
master connection before trying to add it to a master brigade.
> Buckets are also allocated from pools, and pools have different lifetimes depending on
what they were created for. If you allocate a bucket from the request pool, that bucket will
vanish when the request pool is destroyed. Buckets can be passed from one pool to another,
that is what “setaside” means.
> It is really important to get the pool lifetimes right. Allocate something accidentally
from the master connection pool on a slave connection and it appears to work, because generally
the master outlives the slave. Until the master is cleaned up first, and suddenly memory vanishes
unexpectedly in the slave connections - and you crash.
> There were a number of subtle bugs in the proxy where buckets had been allocated from
the wrong pool, and all sorts of weirdness ensued. Make sure your pool lifetimes are allocated
correctly and it will work.

This is all true and correct - as long as all this happens in a single thread. If you have
multiple threads and create sub pools for each from a main pool, each and every create and
destroy of these sub-pools, plus any action on the main pool must be mutex protected. I found

Similar with buckets. When you create a bucket in one thread, you may not destroy it in another
- *while* the bucket_allocator is being used. bucket_allocators are not thread-safe, which
means bucket_brigades are not, which means that all buckets from the same brigade must only
be used inside a single thread.

This means for example that, even though mod_http2 manages the pool lifetime correctly, it
cannot pass a response bucket from a request pool in thread A for writing onto the  main connection
in thread B, *as long as* the response is not complete and thread A is still producing more
buckets with the same allocator. etc. etc.

That is what I mean with not-thread-safe.

View raw message