apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Pane <brian.p...@cnet.com>
Subject Re: [PATCH] Update to Brian's patch to allocate brigades out of the bucket allocator
Date Sat, 21 Dec 2002 07:10:58 GMT
On Fri, 2002-12-20 at 21:37, rbb@apache.org wrote:
> On 20 Dec 2002, Brian Pane wrote:
> 
> > On Fri, 2002-12-20 at 13:50, rbb@apache.org wrote:
> >
> > > > Basically, if you received a brigade from a higher filter, you can only
> > > > assume that it will survive a single trip down the filter stack.
> >
> > Right.  And you also can't assume that the context that passed
> > the brigade down the filter stack still exists by the time the
> > brigade reaches the last filter.  It's a safe assumption in
> > httpd-2.0, where the brigade is passed as a synchronous function
> > call, but it's not valid for apps in general (potentially including
> > future Apache versions) where the brigade may be queued up for
> > asynchronous processing by an I/O completion thread.
> 
> Actually, it's not even a safe assumption in httpd-2.0.  You have data
> from requests that originally lived in brigades allocated out of the
> request pool moved to brigades allocated out of the connection pool
> because the request pool is going away.  But that just proves the point,
> the data must be moved from one brigade to another to change it's
> lifetime.  That is part of the design of buckets and brigades.  It is why
> the concat and split operations were written for maximum performance.
> 
> We are talking about a couple of pointer assignments for the concatenation
> and a malloc and some pointer assignments for the split.

And you're also talking about needing a pool from which you
can do the alloc.  With pool-based brigade allocation, there
are two basic choices when allocating a new brigade during
the split, and they both have problems:

   - Allocate from a connection pool.  This is what httpd-2.0
     does, but it's not a robust solution, given that connections
     can be very long-lived.  And for non-http connections, like
     an app server connector, connections may live forever, so
     doing per-request allocations from a connection-level pool
     guarantees a memory leak.

   - Create some other pool that's guaranteed to have a sufficiently
     long lifetime to accommodate the response.  This isn't a
     viable option either, because creating pools is expensive
     in terms of both time and memory usage.  In addition, the
     app must then ensure that the pool gets cleaned up in all
     error conditions; the cleanup burden simply gets shifted
     from the brigade to the pool.

Pools just aren't a good match for the allocation needs of
brigades.  We happen to have been able to hack around many
of the problems through liberal use of the "copy the brigade
into another pool with a more appropriate scope" technique
in httpd-2.0, but that's not a solution for apps in general.

Brian



Mime
View raw message