httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <gst...@lyra.org>
Subject Re: cvs commit: apache-2.0/src/main http_core.c http_protocol.c
Date Wed, 04 Oct 2000 17:19:16 GMT
[ btw, what is up with you and Bill? Your posts always fall off on the right
  hand side; can you guys try and fit into an 80 column email? as it is, I
  see 80 chars, a linewrap, then the hard CR; it is really hard to read :-) ]

On Wed, Oct 04, 2000 at 04:40:23PM +0000, Greg Ames wrote:
> rbb@covalent.net wrote:
> > > Yes, the cost of allocating/destroying buckets could be greatly reduced.  But
it still
> > > costs something, and you can't avoid setting some fields in each bucket.  Those
> > > operations are likely to cause cache misses when the buckets are first stored
into,
> > > then again each time a filter references them many nanoseconds later.
> >
> > Why would we have a cache miss when referenceing the bucket?  I expect a
> > cache miss the first time we store into a bucket, but not when referencing
> > the data that we stored.
> 
> This may be beating a dead horse somewhat, so I'll try to be brief.  When Bill started
> working on this, he was seeing literally thousands of bucket brigades for a index listing
> of a directory with a fair number of files.  For a directory with one file, he showed
me
> over a hundred, each representing legitimate data much to my surprise.  With HTTP 1.1,
half
> of these are for chunk headers.

Yah, this is that stupid problem that I mentioned a while back about
generating a bazillion chunks because we do it based on each brigade
delivered to the chunk filter, rather than on the buffered/coalesced
output. I saw this with mod_dav output, sending a multistatus response.

> If we buffer at the top of the filter chain but don't coalesce until the core filter,
> nearly all the chunk headers and the corresponding bucket structures go away with little
or
> no additional work.  But we still have on the order of a thousand buckets left for a
> directory with a bunch of files.  On each ap_r* call we allocate a bucket structure,
> initialize it, and hook it into the brigade. Then we unwind back to mod_autoindex's
> handler.  It does it's thing and generates the next little string, then calls ap_r* again.
> When it finally gets done generating the page, our hypothetical buffering code at the
top
> of the filter chain sends the brigade down.  Each filter runs thru the brigade, starting
> with the first bucket.  But by now the first bucket is ancient history to the cache and
we
> are likely to miss.

Two issues: caching and number of trips through the filter stack.

I'm presuming that we would do some coalescing, which solves the former
(which also presumes that it is an actual problem in this case). Regarding
trips through the stack: do we really have hard data on whether that is a
problem? What is a typical filter stack, and what kind of overhead are we
looking at? Knowing that, then we can answer whether we also need to
buffer/coalesce at the top.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Mime
View raw message