Mailing-List: contact new-httpd-help@apache.org; run by ezmlm
Precedence: bulk
Reply-To: new-httpd@apache.org
Date: Thu, 22 Jun 2000 19:37:26 -0400
Message-Id: <200006222337.TAA07810@k5.localdomain>
From: Jeff Trawick <trawick@ibm.net>
To: new-httpd@apache.org
In-reply-to: <38.7992bc5.2683f2f2@aol.com> (TOKILEY@aol.com)
Subject: Re: PLEASE READ: Filter I/O
Reply-to: trawick@ibm.net
References: <38.7992bc5.2683f2f2@aol.com>

Beware... don't bank on my knowledge of HTTP :)

> From: TOKILEY@aol.com
> Date: Thu, 22 Jun 2000 18:53:38 EDT
> 
> If the HTTP protocol had some kind of well-established
> EOD ( End of Data ) signal that could be used as an
> alternative to 'Content-length' then that would alleviate
> the Catch-22 that plauges it ( Not being able to know
> a 'Content-Length' to put in the first buffer until the last
> buffer has been processed ) but it doesn't... and that's
> not going to change.

Chunked encoding takes care of this problem.  In HTTP 1.1, if you
don't have content-length (or you don't wanna) you use chunked
encoding.  Prior to 1.1, the end of transmission (FIN) signals EOD if
there is no content-length.

> So in ANY worthwhile filtering scenario I see the largest
> issue as being this...
> 
> How the heck is my filter supposed to do its thing and
> still have the chance to update 'Content-Length' ( and
> add/subtract any other headers its wants to ) before
> ANYTHING gets 'sent back'.

It is o.k. not to know the content length.  We don't always know
content-length today.  Look at the 25-line comment in the
ap_set_keepalive() function for more information.  Any time we're
doing chunked encoding there is no content-length header.

> That being said I think it's obvious that the filtering
> engine itself is going to have to be able to cache to
> disk no matter what. 100MB of data is just too much
> to hold in any kind of memory for too long unless you
> are just going to depend on the VMM to thrash it's
> brains out and do it for you.

There is the working set issue again, and whether it is Apache
explicitly caching stuff to disk or the well-optimized virtual memory
system doing demand paging, it is a problem.

> 
> Yours...
> Kevin Kiley
> CTO, Remote Communications, Inc.
> http://www.RemoteCommunications.com
> http://www.rctp.com - Online Internet Content Compression Server.
> 
> 


-- 
Jeff Trawick | trawick@ibm.net | PGP public key at web site:
     http://www.geocities.com/SiliconValley/Park/9289/
          Born in Roswell... married an alien...