httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: cvs commit: httpd-2.0 CHANGES
Date Sat, 29 Sep 2001 13:08:21 GMT

In a message dated 01-09-29 12:49:00 EDT, Kiley wrote...

> The classic condition you want to be able to handle is this...
>  1. Netscape browser sends up some HUGE amount of POST
>  data ( very common these days with web-apps proliferating )
>  using 'Transfer-Encoding: chunked'.
>  2. May or may not have 'trailers' to append.
>  3. Netscape will also always automatically add the 'out of band'
>  blank CR/LF following the post data to satisfy legacy UNIX cgi
>  that uses fgets() so those CGI IO calls won't get 'stuck'. This
>  'out of band' CR/LF is NOT included in the 'Content-length:' nor
>  is it included in any Transfer-Encoding 'chunk'. Netscape simply
>  'tacks it on' to end of whatever is sent to host.

Forgot to mention that even Netscape will not ALWAYS send 
the weird 'out of band' ( not included in Content-Length or
Transfer-Encoding ) 'extra' CR/LF on the end of a POST so
you can't just check for 'Mozilla' User-Agent like some 
existing servers do. AFAICT Netscape itself won't add the
'extra' CR/LF if it has been set to 'Proxy'. It will only do it
when talking 'straight' to a Website. I suppose the Netscape
folks are assuming that if the target is not a CGI capable
server ( just a proxy ) then there's no need for the 'out of
band' CR/LF or something. Not sure I follow that logic but
the reality is that you won't ALWAYS get the 'out of band'
stuff from Netscape. Some proxies will also 'throw it away'
automatically when handling inline POSTS from Netscape.

What that means is that sometimes there is just going
to be this 'out of nowhere' CR/LF in the input stream that
doesn't belong to anything and it could very well be 
sitting right between 2 pipelined headers. You HAVE to
have code that says 'If we get a blank CR/LF and no valid
header data preceding it then do NOT treat this as the
EOH for a pipelined request... just throw it away.

There is a classic PATCH for IIS Servers that handles this
very thing. What was happening is that ISAPI security filters
were getting the 'out of nowhere' CR/LF from the IIS engine
and treating it as the end of pipelined header even though
there was absolutely nothing preceding it. Since the ISAPI
security filters were 'assuming' HTTP/1.1 and there was
no Host: field then any POST from Netscape would simply
result in the 'out of band' CR/LF producing an 'Invalid request:
Host field required' error page and connection would close.

The IIS patch installs a 'noise' filter into the IIS 'front door'
that simply throws away any blank CR/LFs that appear
after the legitimate end of one client request and the 
proper start of the next one.

Secret: If Netscape ever DOES send the 'out of band' CR/LF
tacked onto a POST then it will ALWAYS appear in the same
packet that holds the end of the POST data and/or the '0' byte
chunk length that ends the Transfer encoding. The 'out of band'
CR/LF will never come 'all by itself' in a separate packet.


View raw message