apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Reid" <dr...@jetnet.co.uk>
Subject Re: SMS stuff
Date Thu, 07 Jun 2001 11:19:37 GMT

> On Wed, Jun 06, 2001 at 05:35:43PM -0700, Justin Erenkrantz wrote:
> > On Thu, Jun 07, 2001 at 01:19:27AM +0100, David Reid wrote:
> > > This isn't possible with pools.  They limit you to a single way of
> > > at your memory regardless of how it was obtained.
> >
> > What I basically said in my email to David was that I could see having a
> All right. This has to stop.
> David, Sander: bring your conversations onto THIS list. You cannot
> to have conversations in private, develop code in the public APR(UTIL)
> and then expect us to /not/ participate (review, vote, tweak, patch).

To be honest the "issues" that have come up have been on the list before
solutions were committed :(  we are al busy people and reading every email
in total detail takes a large amount of time.

The basic reason for holding discussions in private is that they have been
extremely theoretical and don't have any code associated with them, nor will
they ever.  They also tended to be long and drawn out - not ideal list
material :(

However, point taken.

> The SMS stuff is in APR now, so we are all involved. Keeping conversations
> private is simply going to keep these disconnects happening in the future.


> At the moment, we have a big group of people who see SMS as a substrate
> pools. However, there is a smaller group (David and Sander) who see SMS as
> the end-goal, replacing pools, and that pools will eventually go away.
> How did this disconnect happen? Because you guys talked in private, rather
> than with the rest of the group.

Don't believe that was how it happenned at all.  I was very much a cynic
when I started looking at the sms stuff.  The way that  afew people started
raving on about how cool it all was made me shudder and that was why I
looked at it in such detail.  The reviewing of the code an opening my mind
to the possibilities made me see where it could go.  To be perfectly blunt
Sanders constant evangelising made me more anti sms than pro it.  It was the
code that convinced me it could do a lot for APR :(  But you weren't to know
that as it's not something I've said to people before.

> </rant>
> > pool take in an option as to which SMS it should use.  I've said this
> > before a few weeks ago (probably when I first looked at the SMS code),
> > but let me reiterate that suggestion.  Obviously, there would be a
> > default SMS that would be used, if one isn't provided when the pool is
> > created.
> Exactly. This was my thought, too, so seeing a pool go *inside* of an SMS
> system is whacky.

Why?  sms and pools allocate memory.  A pool is just a specific system for
allocating that at present APR uses throughout, hence we're just adding more
capability to sms.  Don't really see why that is whacky.

> >...
> > IMHO, using APR without something that implements cleanup routines (like
> > what the pool does now) is going to be troublesome to implement without
> > rearchitecting the guts of APR.

Sms has more options for cleanup's than pools.  Sander added a scope
variable so we can do selective cleanup's of children or even within
children.  Given that sms also has a well defined parent/child structure
then this should make our memory easier to cleanup, not harder.  I will
admit that not a lot has been to develop that side more, but the code is
still only a month or so old...

There has never been any notion that sms wouldn't have cleanup's so I'm not
sure where this came from.  Perhaps because there is an apr_sms_free

> Right on the money. APR is built around pools. There is no reason to even
> attempt to believe that it will be built any other way for a while. APR
> could have a different system, but for the next year... we're using pools.

This isn't/has never been in dispute.  I wouldn't consider replacing pools
in the near future and am -1 on replacing them before we have a release of
apache 2.0.  sms is a longer term plan.

> ----
> David said that SMS gives us flexibility that pools don't have. He
> a few things:
> > You want a single block of long lived memory, great use the standard
> > allocator.
> > You want high performance, low footprint memory, use a pool like system.
> > You want simple shared memory, use a shared meory sms.
> > You want higher performance shared meory, use a tracking sms as a child
of a
> > shared sms.
> > etc etc
> Single block of long-lived memory: the global pool (or some other
> pool)
> High perf: use a pool
> Simple shared memory: a pool which uses a shared memory SMS for allocating
> its memory.
> High perf: already have it

Well, we'll agree to differ on these.  The basic problem is that they're all
the same method for allocating memory, all that varies is where the memory
comes from.  Hell, if that's all we want to do then why are we bothering
with sms?  sms adds a lot of baggage if all we really want is to be able to
slot in a backend!

> For the few times when we need to alloc/free at a granular level, then
> apr_pool_create() / apr_pool_destroy() are more than sufficient.

So we create the lists, destroy the lists, all just to free a few bytes.
Seems clumbsy  and overkill to me.  A pool is a whole management system, not
just a piece of memory.  That system has overhead which isn't always needed.

> I am not seeing a large justification for transforming the entire APR
> library (and bubbling that up into Apache, Subversion, etc). The SMS
> alternative systems seem to be able to fit under the pool abstraction.

Well, if that's all we'll ever use sms for then let's ditch it now as it's a
distraction. sms offers much more than that and we don't need it.

> Now, it is abundantly clear that a few of us are seeing it this way. What
> don't understand is what forces it to go the other way? Why does SMS need
> be the top-level abstraction? A simple explanation here would go a long
> For example, "you simple cannot do <X> with a pool, but you do it <THIS>
> with SMS."

Well, I'll try, but to be honest your views seem to be pretty well
entrenched :(  However, I'm in "glass half full" mode so we'll see :)

- every time we create a pool we have to go through and lock the
alloc_mutex.  the code is designed for genral usage and this is required.
With the lock changes this will get faster, but it'll still be there.
Justin pointed out that during connections we really are in a single thread,
so no locking is required.  Also at the end of the connection we'll ditch
the whole thing and start over.  Why not have a simple sms module that
doesn't have any locking and simply allocates memory directly from a single
large lump?  We don't even nee to track it as at the end of the connection
we'll just call reset and memset the whole block to 0.  It should be faster
than pools.
- Cliff says that for the bucket code they want a different way of getting
memory, using thread id's.  In pools we have to allocate a pool and then use
a table or some such to associate blocks from that pool with thread ids,
whereas with sms we simply write a sms module that has the management of
thread ids built in, vastly simplifying things for the bucket code.

2 examples that have come up in the past 24 hours...  They may not be the
best ones, but they are ones that have been suggested/implied by people
other than Sander/myself.

I gues what I see as the big advantage of sms is that it gives you a blank
sheet of paper for your memory.  You can write a very specific sms module
(as will be done for the buckets) with all sorts of optimisations/features
that pools just don't let you.  Pools always control the memory in a
particular way.  They have a set structure and that can't be easily changed.
You want to have refcounting for each allocation - implement it on top of
your pools.  In sms you want refcounting, write the code and integrate it
entirely.  You want specific functions for getting at it, not a problem just
add them for your sms module.

That's why it's important that the locking is done at sms level.  If you
don't need locking you'll uses a non locking sms module.  Why go passing in
variables when we don't really need to?

As the author of a piece of code you should know what sort of memory
management you need.  We'll hopefully end up with an array of modules that
you can choose from, an if the one you want isn't there, write a new one!
Whilst this is possible with pools you're adding another layer on top, with
sms you're not.

> Point out that requirement, and you can change a lot of minds here. But
> until then, I think you'll continue to see confused/concerned people, not
> understanding why you are suggesting we toss all of the memory management
> APR in favor of SMS.

Well, I'll try, but I guess if this horse is dead then we may as well bury
it :(  I'd hate to think that we'd do that at this stage as more work should
be done to try and justify the potential that some of us see in sms.  That
said if the door has already been bolted then the effort that would be
wasted in developing sms further would be just that, wasted, so we should
move on...  Oops, was this really a "glass half full" statement... :=)


View raw message