apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luke Kenneth Casson Leighton <l...@samba-tng.org>
Subject Re: APR shared memory requirements.
Date Thu, 10 May 2001 10:43:38 GMT
On Wed, May 09, 2001 at 06:48:42PM -0700, dean gaudet wrote:
> 
> 
> On Wed, 9 May 2001, Luke Kenneth Casson Leighton wrote:
> 
> > my point is that if you _don't_ #define POOL_DEBUG, this _isn't_ a
> > problem???
> 
> nope -- the ap_pool_join() is a promise by the caller that they won't
> destroy pool B prior to destroying pool A.  well, if you think of this in
> terms of 1.3, which never shares pooled data between threads, what i'm
> really saying is that none of the structures (such as tables) which have
> data allocated in both pools A and B won't be accessed after B has been
> destroyed.
 
so, it's a way to avoid having to do reference counting.

> typically B is a sub-pool of A even in these cases, but sub-pools can
> theoretically be freed before the parent -- and if you have a table
> allocated in A which has pointers to data in B then this results in memory
> corruption.
 
okay.  so you have two separate pools.  they even come from different
management structures.

example.  you have one apr_pool whose memory is allocated from an
apr_memsys that does page-locking for you, guaranteeing that the
memory never hits disk.

you have another apr_pool that allocates from a standard apr_memsys.

and, maybe you have another apr_pool that allocates from a shared
memory apr_memsys.

you _know_ that these three alien memory systems are distinct
and separate, yet for some reason, you use the standard pool
to store a large linked list (or table) of your precious page-locked
pool data (which may be GPG private keys or some-such), etc. etc.

for each memory system, which have destroy methods on all allocations
by the way, you must explicity call the apr_memsys_destroy() when
you are totally finished with them.

and you explicitly do so in the correct order that guarantees the same
promise as apr_pool_join(), except one level above.

> so the debugging code is there to test that when you do this A/B thing
> that you either get a warning, or the code in question needs to make the
> "ap_pool_join promise".
> 
> if you look at the ap_pool_join code you'll see that the side-effect it
> has only modifies data which is present when POOL_DEBUG is active.

ack.  very impressed.  not only were you able to answer my question
which was formed from unclear understanding and issues i've only
heard of, but also to back it up in a way that will reassure others
who understand this clearer than i.

> > uhh, they may be hierarchical, however they all allocate from the
> > same memory, using a static 'management' list.  the 'management' memory
> > for pools most definitely is _not_ hierarchical.
> 
> the block list you mean?  yeah, no hierarchy there.  (and it's strictly
> not required by the pool API, it's a behind-the-scenes optimisation.)
 
ack.

> > ...or, maybe, a better word to use here is 'stackable', not 'hierarchical'.
> > [a la apr_memsys: stackable memory system we're putting together].
> 
> in OO lingo you mean inheritance right?
 
... not quite.  imagine replacing the free and malloc routines in
apr_pool.c with library routines (apr_memsys library routines,
to be more precise).

then, imagine that those library routines are actually ap_pool_alloc 
and friends :)  

[it's a little more involved than that, but you get the idea :) ]

my concern is that apr_pool_join() is going to get in the way
or going to get very confusing.

> > therein lies my difficulty with this.  if you consider doing
> > 'stackable' memory systems, where you have two pools that
> > allocate from two totally different memory systems, they can _not_
> > be apr_pool_join()ed, end of story.
> 
> why not?
> 
> you can still promise that you won't free pool B before you're done using
> cross-pool structures.  ap_pool_join() is a statement about the lifetime
> of those two pools (and this is what my comments in the code try to
> explain).

see above: when one pool is allocated from a totally different
set of 'management' structures - no static lists any more in apr_pool.c,
it's in a 'management' stackable memory system instead - then
the thought of someone trying to do an apr_pool_join() on two
utterly alien apr_pool_t instances just...

i don't know what to suggest!  if you can live with it, fine.
if not, apr_pool_join() should really die.

> of course it's pretty hard to make this promise if one of the pools has
> global lifetime, and is accessed by multiple threads.  this is possible in
> 2.0 now isn't it?  i think that's a mistake if so :)  amongst the many
> features of 1.3 pools were that they were single-threaded access, zero
> mutexes required on the fast path.
> 
> but if one of the pools has a global lifetime, and the other doesn't, then
> you shouldn't be making the ap_pool_join() promise, and so there's no
> problem with the function.
> 
> if you want to continue to support POOL_DEBUG (i am really so far out of
> date with 2.0 internals now that i can't make a judgement on how useful it
> is), then i suspect what you need to do is provide a private function
> which, when given a void *, returns what pool it belongs to.  this would
> be implemented by all the pool classes.
> 
> that handles the POOL_DEBUG stuff which is in the union block_hdr
> structure.
> 
> the "joined" stuff shouldn't need to change, it should be same for all
> pool implementations.
 
okay.  been thinking about this.

maybe this isn't such a bad idea.

... in the light of the above, where apr_pools may be totally
independent and utterly alien to each other, how could it be
implemented?


> note:  i personally think that pools made sense in an http server for
> a few things (the fast path through static responses, and that's about
> it), but don't make any sense at all in pretty much every other program
> in existence, and i kind of question the abuses they're going through.

it makes a lot of sense in dce/rpc.  also in samba TNG.


Mime
View raw message