apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Pane <bp...@pacbell.net>
Subject Re: APR pool maintains too much free list
Date Mon, 28 Jan 2002 16:04:18 GMT
Sander Striker wrote:

>>>I noticed this while looking at Subversion issue #602.  It
>>>is about Subversion consuming too much memory when importing
>>>large tree.
>>>http://subversion.tigris.org/issues/show_bug.cgi?id=602
>>>
>>>I found that memory consumption doesn't go too high if I
>>>compiled APR with --enable-pool-debug so I glanced
>>>memory/unix/apr_pools.c.  There I found that non-DEBUG build
>>>does not free any memory unless the pool holding the
>>>allocator is destroyed or cleared.  I think it should free
>>>memory if it already has enough memory in free list.
>>>
>>This is the point of pools.  The idea is that you should hit a steady
>>state quickly.  Basically, one request goes through, and it allocates
>>all of the memory out of pools.  The next time that same request is
>>sent, it should use the same amount of memory.  For all other requests,
>>it should either use a little more or a little less memory, but at some
>>point you will get a request that uses more memory than any other
>>request, and that is how large your pool will be forever, which means
>>that you will no longer allocate memory.
>>
>>If your pools are growing too large, then you most likely need to split
>>the allocation into multiple sub-pools, so that the memory is returned
>>and can be used by later operations.
>>
>>Ryan
>>
>
>There is that, although I must admit that I have a 'hi free'
>patch lying around.  The idea is to free() all mem on the freelist
>when the freelist size goes over a certain threshold (like a few
>MB).  I would need some feedback on this though,
>

The high free threshold would be useful in at least one
place that I can think of in Apache.  If we change the
worker MPM so that each worker threads can own a persistent
pool (and clear it, rather than destroying it, after each
request), then it would be good to have a mechanism that
sets the max size of a pool's free list following
apr_pool_clear().  E.g., I'd want to configure each ptrans
pool to keep three 8KB blocks in its allocator, for a total
of 24KB, in order to avoid mallocs on the next request most
of the time; but if the last request allocated 10MB within
the pool, we should return most of that space to the system
heap.

But IMHO the only time when we should apply the high free
threshold is when destroying or clearing a pool.

--Brian



Mime
View raw message