httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Leggett <minf...@sharp.fm>
Subject Re: svn commit: r717867 - in /httpd/httpd/trunk: CHANGES docs/manual/mod/mod_buffer.xml docs/manual/mod/mod_buffer.xml.meta modules/filters/config.m4 modules/filters/mod_buffer.c
Date Sun, 16 Nov 2008 22:28:26 GMT
Ruediger Pluem wrote:

> Thanks for clarification. Nevertheless we should make it more clear in the
> documentation that this module can make performance worse if used in the
> wrong context, e.g. with static files. Otherwise users might think that
> buffering files via mod_buffer could be a cool performance boost which
> it is not.

I have added a warning to the docs to better explain just what the 
buffer does, and that it is not going to help in cases where the 
response is already efficient.

> You can have a mixture of both because there can be include files in your
> nested include files that have a larger chunk while many other are as small
> as you describe. IMHO by taking care of this case you don't loose any of your
> buffer features as the resulting brigade is nearly the same with the difference
> that we just recreated and mem copied the buckets we had before. The only
> difference that matters appears in the case of a small file bucket that did
> not fit in the previous heap bucket.
> IMHO you put too much burden on the administrator to know the internals of
> httpd in order to make a good decision whether to use mod_buffer or not.

In theory the administrator can try mod_buffer, and if it makes it 
faster, all well and good, if it doesn't oh well.

I took quite a while looking at this, most specifically a potential 
optimisation where if the heap bucket you were adding was already 8k in 
size, just add the bucket and don't repack. But whichever way I looked 
at it, I could see potential for the optimisation to be lost.

I think further optimisations should probably come with switches to 
control how they work.

>> The point behind the buffer is to convert whatever buckets are present
>> into the smallest number of heap buckets as possible, and do this until
>> the buffer is full, or the response is complete, whichever comes first.
>>
>> Think of a potentially expensive bucket of known but long length - we
>> certainly don't want to blindly move the expensive bucket over in this
>> case, as the bucket will only get read at some point in the distance
>> future when the slow client gets round to read the bucket, and the whole
>> purpose of buffering is lost.
> 
> Could you please elaborate of which kind of expensive bucket you are talking
> about?

One example is data reverse proxied from an expensive backend and then 
built into a page using mod_include. By reading the full response in 
before doing the blocking sending it to the client, you give your 
backend a fighting chance to release resources immediately rather than 
ages later when the network eventually gets round to accepting the data.

When the proxy is optimised to release the backend connection when the 
EOS is read from the backend, rather than after the EOS is successfully 
delivered downstream, this will help for normal reverse proxied data too.

> My assumption is that we deal with heap buckets in most cases, but anyway
> this problem can be solved by checking
> 
> APR_BRIGADE_EMPTY(ctx->bb) && APR_BUCKET_IS_HEAP(e)
> 
> and proceed in the current way if this is not true.

I looked at this again, we can extend the optimisation beyond just the 
first bucket on condition the buckets stay heap buckets, and stay 
APR_BUCKET_BUFF_SIZE in length. This way if someone buffers a file (or 
other potentially large bucket), the copy is avoided, and we stick with 
a move.

Regards,
Graham
--

Mime
View raw message