apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Reid" <dr...@jetnet.co.uk>
Subject Re: cvs commit: apr/threadproc/unix thread.c
Date Thu, 07 Jun 2001 00:19:27 GMT
I've just replied to Justin direct, but I'll say it here as well...

We could implement pool using sms but we'd loose a great deal of flexibility
and a great opportunity to make APR even more useful.

Pools are a single way of dealing with handing out memory.  they imply a
degree of overhead and while they work eveyrwhere there is some degree of
agreement that they're overkill in a lot of cases.  So why do it?

SMS gives us a very great deal of flexability.

You want a single block of long lived memory, great use the standard
allocator.
You want high performance, low footprint memory, use a pool like system.
You want simple shared memory, use a shared meory sms.
You want higher performance shared meory, use a tracking sms as a child of a
shared sms.
etc etc

This isn't possible with pools.  They limit you to a single way of getting
at your memory regardless of how it was obtained.

As each sms has the opportunity to control exactly how it locks, allocates,
frees and so on they have enormous potential to be varied and written for
specific purposes, each one optimised.  This means we have a memory system
that can be tailored, so one size doesn't have to suit everyone.

Take apache...

top level pool doesn't really change much, is only alloacted at startup, so
it could be a simple, low overhead standard sms.
pool for connection data - rapidly changing, performance more important than
overhead, use a tracking sms.
ssl certificates, needed by all processes but don't change very often,
simple shared sms

Imagine a crude caching sms.  You create it with a default time for each
allocation.  It could start a thread that cleaned up the memory when the
time expired, all without the app needing to be bothered beyond it's initial
allocation.  Do we want such a thing, probably not but it could be
written/used easily using sms, more difficult using pools.

What's the difference to pools?  You are in control of how the memory is
used.  Pools don't really give you that.  do we need that flexibility?
Well, given we have the opportunity and my magic ball is out of warranty,
I'd say Yes.  Justin it would appear says No.

Time for bed.

david

> > On Wed, Jun 06, 2001 at 11:47:53PM +0100, David Reid wrote:
> > > This is the crux of the issue methinks.  We don't yet have a module
that
> > > would allow us to even get close to replacing pools.  We need a lot of
> > > things from it and Sander and I have had some good early
> > discussions about
> > > how it could work.  Basically we want to have a fast, stable tracking
> > > allocator that has a smaller memory footprint than pools.  Is
> > it possible?
> > > I don't honestly know but we're going to give it a good try.
> > Why haven't we
> > > opened up our discussions?  Because we haven't even got any code and
are
> > > still bashing around the early design which is probably better done
> > > privately.  Once we have something we like we'll post.
>
> I agree.
>
> > See, I think this is the difference.  I see that the pools are on top of
> > sms.  (Gee, this is what Cliff said...)  The sms doesn't need to know
> > anything about refcounting or anything special.  What does refcounting
> > give you?  I'm still not also sure why locking needs to be in the SMS.
> > (I think I asked for clarification on this, but I received none...)
>
> See below and previous messages.
>
> > All an sms knows how to do is to get a chunk and free a chunk of
> > memory.  None of the pool logic needs to be in sms.  I saw that the sms
> > were just an abstraction around allocating memory.
>
> And cleanups... And some other stuff that is in the works.
>
> > The pool will
> > actually handle the cleanups.  Everyone still uses apr_pool_t.  The
> > pool itself uses apr_sms_t to allocate memory.
>
> Which gives us another level of indirection that is just not necessary.
> The sms framework can already pretty much handle what pools can handle.
> There are just a few sms implementations to be written like the ones
> you mention below.
>
> > This enables us to have
> > a shared memory pool, a file-backed memory pool, a heap-backed memory
> > pool - whatever we want.
>
> Replace pool with sms in the previous lines and you are right on the
> money :-)
>
> >  The sms doesn't need to do any locking - the
> > pool will guarantee that the allocation is done atomically by its own
> > locking mechanisms (what it does now - albeit the pool locking is a bit
> > coarser than it really needs to be).
>
> If this is the general consent we might aswell have added a few function
> pointers (for alloc/realloc/free) to apr_pool_t and be done with it. We
> wanted to do the entire thing without being held back by previous design.
> Also, sms was designed independent of apr, it was inspired on glances on
> the apr library (mostly the documentation :-).
>
> > I think the thing is that I've seen the sms as slightly different than
> > what it was originally posted as.  So, I might be in the minority here.
> > I think we are seeing two different views of what an sms should be.
>
> Yes, I think we need to get this sync'ed up. However, this seems to
> take some time, so implementing some things on our side and then showing
> what it does, why and how it is supposes to be used can be dealt with
> after that. I really don't like selling air :-)
>
> > My -1 was non-veto, so it doesn't stop you.  It just registers my
> > dissent.  -- justin


Mime
View raw message