httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <gst...@lyra.org>
Subject Re: cvs commit: httpd-2.0 STATUS
Date Fri, 12 Jan 2001 20:37:49 GMT
On Fri, Jan 12, 2001 at 12:32:23PM -0800, rbb@covalent.net wrote:
> 
> > >   +    * Mod_autoindex is still causing too many buckets and too many bucket
> > >   +      brigades to be created.  We need to improve the way the old ap_r*
> > >   +      functions interact with buckets.  This is being tabled until after
> > >   +      the beta.
> > >   +          See MSG: <Pine.LNX.4.21.0101111403150.1557-100000@koj>
> > 
> > I had a whacky idea about this a little while ago.
> > 
> > Consider: our main problem is the allocation of the brigade and the
> > transient bucket. We use the transient bucket to defer copying until
> > necessary. That is an important part, so we shouldn't lose that.
> > 
> > Consider: ap_r* are NOT re-entrant. Across all the functions.
> > 
> > Possible answer: "allocate" a brigade and a custom bucket type in the
> > request_rec. ap_r* would fill in some values and pass the brigade down. If a
> > set-aside occurs, then the custom bucket simply copies into heap bucket or
> > whatever. During a set-aside, the brigade we pass won't be used because the
> > code doing the set-aside can't know the pool the thing was allocated in, so
> > we don't have to worry about the fixed structure in the request_rec. Since
> > ap_r* aren't reentrant, again, we don't have to worry about the fixed
> > structs in the request_rec.
> 
> I'm missing something here.  How does this keep us from creating buckets
> that only refer to a couple of bytes?  I don't think the problem is the
> number of brigades, it is the sheer number of buckets that we end up
> malloc'ing.  If you call ap_r*, you will end up passing a single bucket
> down the stack.  If a later filter calls the set-aside function, then we
> will end up creating a corresponding heap bucket.  Unless the set-aside
> function is going to try to coalesce, this doesn't solve the problem, it
> just moves it from the ap_r* function to a point later in processing.

Agreed that is defers the processing, but the solution at that point will
apply to *all* output mechanisms (ap_r or buckets). Consider mod_include
spitting out a bazillion little buckets (in a big-ass brigade, but small
buckets nonetheless). Whatever is coalescing the ap_r buckets can apply to
the mod_include buckets, too.

So if that setaside is smart and does a coalesce, then we're golden all the
way around.

[ and note that BUFF's strategy was similar: pass in little bits, it gets to
  the bottom of the stack, and gets plopped into one big buffer, pending
  transmission. ]


What I'm trying to say is that we can avoid the massive number of allocated
buckets at the top, caused by ap_r. Dealing with the setaside is a more
general problem, and exists regardless of ap_r. IOW, I believe the above
(possible) solution will reduce a couple problems down to one.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

Mime
View raw message