httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Leggett <minf...@sharp.fm>
Subject Re: buckets and connections (long post)
Date Thu, 22 Oct 2015 16:26:23 GMT
On 22 Oct 2015, at 5:55 PM, Stefan Eissing <stefan.eissing@greenbytes.de> wrote:

>> With the async filters this flow control is now made available to every filter in
the ap_filter_setaside_brigade() function. When mod_http2 handles async filters you’ll get
this flow control for free.
> 
> No, it will not. The processing of responses is very different.
> 
> Example: there is individual flow control of responses in HTTP/2. Clients do small window
sizes on images, like 64KB in order to get small images completely or only the meta data of
large ones. For these large files, the client does not send flow-control updates until it
has received all other
> resources it is interested in. *Then* it tells the server to go ahead and send the rest
of these images.
> 
> This means a file bucket for such images will hang around for an indefinite amount of
time and a filter cannot say, "Oh, I have n file buckets queued, let's block write them first
before I accept more." The server cannot do that.

What you’re describing is a DoS.

A client can’t tie up resources for an arbitrary amount of time, the server needs to be
under control of this. If a client wants part of a file, the server needs to open the file,
send the part, then close the file and be done. If the client wants more, then the server
opens up the file again, sends more, and then is done.

> I certainly do not want to reinvent the wheel here and I am very glad about any existing
solution and getting told how to use them. But please try to understand the specific problems
before saying "we must have already a solution for that, go find it. you will see…"

The http2 code needs to fit in with the code that is already there, and most importantly it
needs to ensure it doesn’t clash with the existing mechanisms. If an existing mechanism
isn’t enough, it can be extended, but they must not be bypassed.

The mechanism in the core keeps track of the number of file buckets, in-memory buckets and
requests “in flight”, and then blocks if this gets too high. Rather block and live to
fight another day than try an open too many files and get spurious failures as you run out
of file descriptors.

The async filters gives you the ap_filter_should_yield() function that will tell you if downstream
is too full and you should hold off sending more data. For example, don’t accept another
request if you’ve already got too many requests in flight.

Regards,
Graham
—


Mime
View raw message