apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dean gaudet <dean-list-apr-...@arctic.org>
Subject Re: cvs commit: apr/memory/unix apr_sms.c apr_sms_trivial.c sms_private.h
Date Sat, 07 Jul 2001 18:11:19 GMT
On Sat, 7 Jul 2001, Aaron Bannert wrote:

> > > I don't think there is any benefit to this.  But, someone else might
> > > be able to chime in and say why we need this functionality.  -- justin
> >
> > how do you handle out of memory?
> >
> > palloc is not permitted to return NULL for performance reasons.  (well
> > that plus the massive amounts of legacy code)
> >
> > of course this whole topic is a mess and really wants proper exception
> > handling... which we don't have in a convenient form in C :)
>
> palloc() will never return NULL? Now I'm curious how out-of-memory is
> handled. I guess I (wrongly) assumed palloc() would return NULL the
> way malloc() does. *shrug*

palloc is not malloc.

just grep in httpd-2.0 for palloc and notice how nothing checks its return
value.  (ok maybe something does, i didn't look close enough.  look at
apache-1.3 if you don't believe me :).

the truth is that it's pointless to check for out-of-memory return values
in a portable program anyhow.  the various sane operating systems out
there use optimistic memory allocation (i.e. linux) because pessimistic
memory allocation (i.e. solaris) sucks ass whenever you work with a
fork()d server.  solaris boxes need massive swap partitions to handle
bloated mod_perl applications which consist of a few dozen fork()s of the
same process -- because solaris wants to commit backing store for every
single page which is marked copy-on-write.

so if you've got, say, 20MB of memory which is shared and never modified
between 20 processes, then solaris wants 400MB of backing store.  linux
won't allocate backing store until a write actually occurs on that page.

this means linux won't give you an "out of memory error" until you try to
write to a page and it can't allocate a page of storage.  how does it
signal the error?  it sends you a SIGSEGV.

i forget where the various BSDs live in the spectrum of optimistic vs.
pessimistic memory allocation.

i also shouldn't rag TOO hard on solaris, there is a way to do optimistic
memory allocation (using mmap()), but that's not what malloc() uses.
it's also in good company, there's other pessimistic beasties out there.

so anyhow, malloc() can give you a non-NULL pointer for storage that just
plain doesn't exist, and you can waste your time testing it's non-NULL
only to get a SIGSEGV when you try to actually use it.

it's better to just abort when you get a NULL pointer :)  you're really in
trouble when that happens.  in order to unwind from a NULL on memory
allocation you would have to have carefully allocated (and written to!)
all the memory (and stack space) required by your recovery mechanism.
nobody codes that well.  you might try to write your own code that well,
but what about all the libraries you use?

now... you guys may have palloc() returning NULL because you've put a
layer of indirection into pools so that you can have pools on top of
exotic storages.  if that's the case, then i really hope that one of those
exotic pools is never passed to an httpd routine expecting a plain old
memory pool.  'cause you'll waste a lot of cpu cycles putting NULL checks
everywhere.

i didn't look to see what happenned where the old abort() was.  anything
which manages to squeeze out a log message (or exit(1) trying) is totally
cool.

-dean


Mime
View raw message