httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: what are the issues? (was: Re: Patch review: ...)
Date Fri, 30 Jun 2000 04:09:26 GMT

In a message dated 00-06-30 02:42:16 EDT, Rasmus Lerdorf wrote...

> >Isn't the more important reason to have a content-length header that
> >browsers won't do keep-alive without one?  I never really understand that
> >limitation though.
>  Roy Fielding responded...
>  An HTTP/1.1 browser will do persistent connections with chunked, but
>  you are right for HTTP/1.0 browsers.  The reason is that, if the client
>  is 1.0, we have to close the connection because chunked isn't allowed
>  and there is no other way to indicate an end-of-content without length.

It's not just browsers that are the issue. Browsers now only make 
up a 'good' part of HTTP traffic... there are cadres of 'user-agents'
and robots and squids and proxies and firewalls and net-nannies
and filters and akamai-style-caching-mirrors and god all what else
that are in the picture here now, too, and cannot be ignored. Any 
one of them can break the HTTP scheme just as fast as a poorly 
coded or not-fully-up-to-spec 'browser'.

I think design discussions should use the term 'user-agents' rather
than 'browsers' because that's the real term for any HTTP client requestor
and it keeps the 'design' issues focused on the broader reality.

Why is this important? Because if you start basing design decisions
for new features in Apache just based on what 'browsers' are doing
you are ignoring the fact that MANY 'user-agents' which claim to
be HTTP 1.1 compliant are no such thing. That's just the way it is.

When HTTP 1.1 came around people hurried to add support and
made sure they could do 'Keep-Alive' solely based on whether
the 'Connection:' header said so or not. They ignored a lot of the
other considerations that related to it such as being able to
do 'Keep-Alive' AND support the 'No content length' 
Transfer-encoding: chunked method of doing same.

>  An HTTP/1.1 browser will do persistent connections with chunked

This is my point. The reality today is that this line should read...

>  An HTTP/1.1 user-agent is SUPPOSED to do persistent connections 
when 'Keep Alive' is authorized by the Server and there is no Content-Length
specified in the response header but the 'Transfer-encoding: chunked' method 
is used... but currently, many do not.

There have been at least 1 or 2 major releases of widely used user-agents
( which happen to be browsers and popular proxy servers ) that are 'saying' 
they are HTTP 1.1 compliant but will no more do persistent connections 
without a valid 'Content-Length:' in the header than they will dance the 

In the same way there have been releases of the same widely used 
products that despite the fact they are saying HTTP 1.1 and sending 
the 'Accept-Encoding: gzip,compress' field they will no more accept
and decompress a 'Content-Encoding: gzip' response than would the
man in the moon himself. ( There's that 'man moon' thing again! ).

...and it gets even worse when you consider the current reality
regarding the 'Transfer-encoding:' field itself. I have yet to find
any major 'user-agent' that is decoding that field correctly 
according to the RFC's when there is more than one comma
separated entry in the field and more than 1 transfer encoding
applied to the transaction. In this specific regard the usage of
the 'Transfer-Encoding:' field at large is, as far as I can tell,
totally broken. Even if you find one that can actually accept
a double transfer encoding ( chunked, gzip ) try doing three
( like you are supposed to be able to ) and watch the fireworks.

Here's a real-world example to consider...

As of this moment... there is not even one major HTTP 
online testing service or benchmarking suite that, despite 
the fact it is saying it is HTTP 1.1 compliant, will accept
a 'Content-Encoding: gzip' response nor will most of
them truly and consistently perform 'Keep-Alive' without 
a valid Content-Length... and BOTH of those things are
in the HTTP 1.1 specification and have been for a long time. 

So what am I saying?

I am saying that CONTENT-LENGTH IS KEY!

If you say you can 'always just use Transfer-encoding: chunked
and still expect people to use the Transfer-encoding field for
ANYTHING other than that or expect persistent connections
to just magically be maintained throughout the endless chain
of touch points that comprise a modern day HTTP request
you are pretty much dreaming.

It's time for the third HTTP EOD ( End of Data ) option to be
there which should have been there in the first place and
that is a TRUE transport-level in-stream EOD like just about
every other communications protocol ever designed has had.

There has ALWAYS needed to be some way for a user-agent
to detect EOD directly in the stream and there are still only
about a thousand good ways to do that.

The only alternative to a true HTTP in-stream EOD or 
an actual HTTP 'footer' is to either fix support for the
Transfer-Encoding field by forcing all 'not-fully-HTTP 1.1' 
compliant user-agents to break in one swell foop or find
a way to make sure Content-Length is ALWAYS there.

If a filtering scheme of any kind is implemented that doesn't
maintain Content-Length and forces the use of Transfer-encoding
for one specific thing ( chunked ) then there are going to be
some major 'power-outtages' out there and a lot of finger-pointing.

...and as the TV commercial says... 'Are you ready?'.


Kevin Kiley
CTO, Remote Communications, Inc. - Online Internet Content Compression Server

View raw message