apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Stein <gst...@gmail.com>
Subject Re: reducing memory fragmentation
Date Sat, 19 Feb 2011 02:34:55 GMT
On Fri, Feb 18, 2011 at 16:55, Stefan Fritsch <sf@sfritsch.de> wrote:
> On Thursday 17 February 2011, Jim Jagielski wrote:
>> Please also look at Greg's pocore memory allocation stuff...
>> his hash stuff is also quite nice. Would be useful to be
>> able to use that as well....
>
> That looks like a much bigger change than what I have just commited.
> But I agree that before trying to optimize apr's allocator, one should
> try pocore's.
>
> Have you thought about how to do this? Probably every apr pool would
> be a wrapper for a pocore pool. But what about the other users of
> apr_allocator, like bucket allocators?

There is a branch in apr for wrapping pocore's pools and hash
tables[1]. Obviously, the indirection slows it down, but it does
demonstrate how it would work. (and it does: I've run the entire svn
test suite using this wrapping)

Just one comment: use pocore's child pools rather than its "post"
notion. I need to get rid of the posts and simply speed up child
pools.

The basic problem with the apr allocator is its granularity. The reuse
of memory is also a problem. pocore will store arbitrary-sized
segments and unused fragments and whatnot into a red-black tree to
quickly grab it and reuse it.

My experiments show that mmap/munmap are NOT speed-advantageous on
MacOS. But if you're looking at long-term memory usage and avoiding
fragmentation... I don't have a good way to test that. That said,
pocore should not be subject to fragmentation like apr. Its coalescing
feature (designed w/ the APIs present, but not yet coded) will avoid
much fragmentation.

Cheers,
-g

[1] http://svn.apache.org/repos/asf/apr/apr/branches/gstein-pocore/
... see memory/unix/apr_pools.c and tables/apr_hash.c

Mime
View raw message