httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graham Leggett <>
Subject Proposal: adoption of mod_policy subproject
Date Tue, 13 Dec 2011 16:22:13 GMT
Hi all,

As with mod_firehose, I have concluded negotiation with the BBC to open source some httpd
modules that I wrote under the AL, and the BBC have very kindly agreed to donate the code
to the ASF[1], which I believe would fit well as a subproject of httpd, and would like to
know whether httpd would accept these.

To be clear, this isn't a "code dump", my intention is to continue to develop and support
this moving forward, and hopefully expand the community around them.

- mod_policy: "HTTP protocol police"

One of the curses of developing restful services is that clients are "liberal in what they
accept". This leads many developers of restful services to be "liberal in what they send",
resulting in a service that works for the developer, but fails under load or other real world

mod_policy is a set of httpd filters that detect and implement a set of HTTP protocol checks,
the idea being you declare a policy for your development and testing environments, and requests/responses
that violate the policy will either log a warning to the error_log or explicitly fail with
a suitable error message, clearly telling the developer what they have done wrong, with the
expectation that the developer fixes this before the code sees production.

The set of policies to apply is as follows, but is expected to change with time:

o Content-Type: check that it's present and valid
o Content-Length: check that it is present and valid (used to ensure that keepalive requests
between httpd and load balancers aren't prematurely terminated by a Connection: close)
o Keepalive: more detailed keepalive checks
o Vary: headers like User-Agent represent a potential caching DoS, if specified header is
present in Vary, fail
o Validation: if ETag/Last-Modified not present, fail
o Conditional: if a conditional request doesn't comes back with a properly compliant conditional
response, fail
o No-cache: if the response is declared no cache, fail
o Max-age: if the response has a max-age less than a given threshold, fail
o Version: if the request was less than a given version (< HTTP1/1, for example) fail

These are an initial set of policies that were created to meet current needs at the time of
development, however it is expected this list will grow with time. mod_policy would benefit
greatly from the experience of the authorities on HTTP that exist here, with the above policies
being tightened up and improved.

With the proliferation of restful services out there in various states of dubious protocol
compliance, this set of filters can be a huge help to stop developers doing non compliant
things, while not getting in the way of production code. The filters also help enforce that
content remains cacheable, which for sites that endure high loads or thundering herds is important.

The code is currently packaged as an RPM, wrapped in autotools, and a snapshot is available

The corresponding README documenting in more detail is here:

The code itself is here:

Obviously the expectation is for the documentation to be completed and fleshed out.



View raw message