apr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Fuhrmann <stefan.fuhrm...@wandisco.com>
Subject Re: [PATCH] APR pool scalability and efficiency
Date Sun, 11 May 2014 14:07:32 GMT
On Sun, May 11, 2014 at 10:28 AM, Stefan Fritsch <sf@sfritsch.de> wrote:

> Hi Stefan,
>
> On Sunday 11 May 2014 04:23:38, Stefan Fuhrmann wrote:
> > The first and more important one was getting an OOM
> > error because we hit the mmap region count limit on
> > Ubuntu (mmap seems to be enabled in the system libs
> > at least since 12.04 LTS).
>
> Using mmap has the advantage that free memory can be given back to the
> OS much more often, which is a big advantage for httpd, especially
> with mpm prefork. Therefore I enabled use of mmap in Debian/Ubuntu.
>

It's perfectly reasonable to use mmap and at least under
Linux, there is a way to bump that region limit to whatever
an application could possibly need.


> > Its root cause was a large data
> > structure (many small allocations) being built up in a single
> > pool. That times multiple threads / concurrent requests
> > exceeded the ~0.5GB limit (~64k regions x 8kB).
> > [apr-pool-growth.*]
>
> interesting. From my experience, Linux seems to merge adjacent
> anonymous mappings. How does the process map look when it goes OOM?
> Does subversion create lots of file-backed mappings that are
> interspersed with anon mappings? Or does the OOM happen when one
> thread unmaps its memory, causing a lot of fragmentation?
>

Nope. Although the original code contains a few temporary
subpools, the following code already exhibits the problem
when run in the SVN test suite:

static void test_pools(apr_pool_t *pool)
{
  int i;
  for (i = 0; i < 10000000; ++i)
    {
      if (i % 100000 == 0)
        printf("%d\n", i);
      apr_palloc(pool, 100);
    }
}

Output:

...
5000000
5100000
libsvn: Out of memory - terminating application.
*** Program received signal SIGABRT (Aborted) ***

So, it seems that the individual regions do not get combined.
IIRC from earlier debug output, they are actually adjacent, though.

> In a related scenario, we would allocate a large buffer
> > (e.g. 64kB) in a temporary pool, clear the pool and then
> > create another pool that is small (few allocations) but
> > relatively long-lived. The APR allocator would then tend
> > to reuse the large node for the new pool - wasting quite
> > a bit of memory. [apr-allocator-efficiency.*]
>
> I have thought about this case, too. A different approach would be to
> split the large node, take what was requested for the current
> allocation and put the rest back into the free list as a smaller node.
> This works only with mmap/munmap and not with malloc/free, of course.
> Not sure which method is better. One could do both, I guess.
>

Yes, one can make a case both ways. Splitting nodes gives
"tighter" memory usage with less portability. Stricter allocation
preserves the large nodes for when they are needed, possibly
reducing fragmentation, while risking them being released to
the OS as we hit the allocator's free memory limit.

-- Stefan^2.

[Once again showing that numbering one's name resolves ambiguity.]

Mime
View raw message