httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Eissing <>
Subject Re: mod_proxy_http2 questions (was: [VOTE] backport mod_proxy_http2 to 2.4.x as experimental)
Date Thu, 10 Mar 2016 16:38:25 GMT
Hi Yann,

thanks for the questions. I'd really love to bounce the idea of my implementation
around here, because I hope that I got things right and, since I do not know all
corners and hooks in our little product, to get feedback what I have overlooked.

Iterative, the Common Case
First, if mod_proxy_http2 is called in a HTTP/1 connection, all should be pretty
much the same as for mod_proxy_http. I basically started with that code and added
the HTTP/2 session handling in. 

The HTTP/2 session is allocated with proxy_conn->connection->pool and registers
there for shutdown. So when mod_proxy_http2 get called serial for one request
after the other, it will find an existing HTTP/2 session on an already used
connection and use that again. That is nothing different than the HTTP/1.1 case
with some additional state piggy backed on the backend connection.

This was the first iteration of the module.

As to the input/output handling for that request_rec, that is basic mod_http2
stuff. The core filters have been replaced with ones that shuffle the data
to h2 internal things (mainly through the h2_mplx) and the response headers
are taken directly from the request_rec->headers_out. The http2 usual way.

Concurrent Handling
I was then thinking: how do I get all the concurrent proxy_http2 requests to
use the same connection? I was originally thinking about some dort of de-mux,
a sort of concentrator that sits on a single connection and communicates with
all the worker threads that have proxy requests...

But that would be insane! There would be one worker sitting on the master
connection, several workers for each single proxy request and one more
worker that sits on the backend connection. And all the proxy requests 
would do is move data from the master connection thread to the backend
one and vice versa. But what we need is just 2 workers: one for the master
connection and one for the backend connection. They should handle as many
concurrent requests as configured!

Backend Engines
How can we do that? Let's discuss one master http2 connection with
no proxy request ongoing. The next request which is handled by proxy_http2
is the "first" - it sees no other ongoing request against the same 
backend. It registers itself at the master http2 session for the backend.
It performs the request in the usual way, de-registers itself again
and returns. Same as with HTTP/1, plus some registration dance of what
I named a "request engine".

If another worker handles a proxy_http2 request, while the "first"
engine is ongoing, it sees that someone has registered for the backend
that it also needs. What it then does is "hand over" the request_rec
to the "first" engine and immediately returns as if done.

So, the "first" proxy_http2 worker collects all request_recs against
the same backend and continues running until it is out of requests. Then 
it also shuts down.

So, while several requests against the same backend are ongoing, there
are two main workers handling it, plus, very shortly, some workers that
create request_recs, find out that they are proxy_http2, hand them over 
and return.

So, how do we hand request_recs over to another thread?

Request Transfer
First of all, the request_recs inside a HTTP/2 connection are all created
on top of slave connections and mod_http2 has a tight grip of the lifetime
of those connections. So, all that has been allocated from that slave
connection directly will still be there when the mod_proxy handler

But the request_rec itself and all filters that have been installed
use a child pool and the usual runtime gets rid of that pool at the
end of the request. How does it know that the request has ended? When
the EOR bucket gets destroyed.

So, mod_http2, when handing over request_recs, first *freezes* it
by adding a top output filter that sets all buckets it gets aside. Which
basically is the EOR bucket. It passes nothing on while frozen, so 
output filters further down never get called at this time.

When the proxy_http2 worker for that request returns, mod_http2
notices the frozen request, thaws it and signals the backend connection
worker that the request is ready for processing. The backend worker
picks it up, handles it and *closes the output* which will write
the buckets set aside before. This will write the EOR and the 
request deallocates itself. During this time, all the input/output
filter etc. will remain in place and work.

Conclusion, TODOs
As to you question re filters on the backend connection: those are
used unchanged. However they will be used for HTTP/2 data and while
the byte counters will do, any request counter will not. I do not
know what else is done there...

Also I have not had the time to look at the logging of such request.
There might be some more work done there.

I hope this give an explanation of how mod_proxy_http2 works
together with mod_http2. If not, please ask. Happy to explain and
get feedback.



> Am 10.03.2016 um 16:48 schrieb Yann Ylavic <>:
> Hi Stefan,
> sorry I didn't look closely enough in mod_proxy_http2's code yet to
> answer the following questions by myself, so I'm asking here...
> On Wed, Mar 9, 2016 at 1:53 PM, Stefan Eissing
> <> wrote:
>> When called inside a HTTP/1.1 connection, it will open/reuse
>> an existing HTTP/2 backend connection for this one request.
>> When called inside a HTTP/2 connection, new requests can be
>> transferred to an already ongoing backend HTTP/2 connection
>> for the same master.
> I can follow the request path which traverses the "normal" http/1
> hooks/filters (after the http/2 ones in the latter case), but the
> response seems to be handled solely in the http2 code.
> Am I right to think that any registered request or connection output
> filter will get called as usual with an HTTP/1 response?
> It may not make much sense for things like mod_deflate (already
> integrated in h2) but others like mod_headers/security/third-party
> working at the request level would still want to play here...
> Also, what about filters registered on the backend connection?
> With mod_proxy_http the proxy_create_req hook allows this and for
> example I (will) use it in a patch (about to be proposed) to
> mod_logio/mod_status for per-vhost counters (rated incoming and
> outgoing connections, requests, bytes, TTFB, TTLB, ..., on both the
> client and backend sides, implemented with some (new) mod_logio
> connection filters).
> Since there is no "fake" request_rec used/needed in mod_proxy_http2,
> the is no such hook, but I guess there can be one (or even several
> more specific to h2, dunno)...
> Anyway, that is surely not a primary goal, I'm just thinking out loud here :)
> Thanks,
> Yann.

View raw message