trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Plevyak <jplev...@gmail.com>
Subject Re: ATS freebsd memory usage
Date Mon, 13 Feb 2012 17:01:53 GMT
AFAICT, the RAM cache is working as designed and it doesn't grow
"unbounded".   There may be issues with interactions in the OS and glibc,
but if there are, I would like to understand them.  Here is the last
message on the bug:

> the memory waste in TS is mostly because the Ramcache:
> 1, it is counted by really used memory.

Yes, that is how it is designed.  What else would you like it to do?
 Perhaps the OS memory allocation overhead?  If so, please confirm.

> 2, it will hold the whole block of memory from free to OS, the old glibc
memory management issue.

Not clear what this means.  Yes, it holds the block of memory given by the
OS (when malloc is called), or from the IOBufferBlock freelists.  What else
might it do?

> 3, ramcache use the cache reader buffer, and the buffer is allocated from
anywhere in the whole memory adress

Yes, memory is allocated across the address space.  The OS does fragment
the virtual memory space to some extent when large blocks of memory are
allocated (mmap(0)).  That is normal.  What are you suggesting is wrong
here?

> all those make TS waste much more memory than Ramcache is configured. it
will eat all you memory at then end.

You haven't pointed out any problems.  If there is a problem with the glibc
memory allocator we can move to tcmalloc or any other allocator.  Some
suggested solution here would be nice along with some pointer (e.g. URL) to
a discussion of the OS/glibc problem you are referring to.

> what we want to do:
> 1, limit ramcache memory by allocator

Not clear what this means.  Do you want to restrict the number of say 32K
objects which can be stored in the RAM cache?  This is likely to interact
badly with the LRU nature of the algorithm.

> 2, bound ram lur list and freelist

The LRU list is bounded by number of bytes in the objects it contains.  Are
you suggesting that you are getting a flurry of very small objects which is
causing the number of entries to jump and then that memory is not free'd
back to the OS?  What use pattern would cause this?  Is this a real problem?

> 3, freelist will be split by size

which freelist are we referring to here?  the entries freelist, the
iobuffer freelists are already split by size.  What reason could there be
to split the entry freelist by size?  in RamCacheCLFUS::victimize() the
data field containing the smart pointer to the memory is cleared, at which
point any interest in the size of the data associated with the entry should
have ended.

> 4, split ramcache memory and cache memory

The ability to share memory between the ram cache and the iobuffers used to
deliver content permits zero-copy RAM cache hits.  What would you gain by
throwing that away?

Some clarification would be great.  Clearly memory fragments in a large
system, but I don't see any documentation of those effects or the extent of
the problem attached to this bug.

> the codes will be ready in days

I am curious to see what code this might be.

On Mon, Feb 13, 2012 at 7:41 AM, Leif Hedstrom <zwoop@apache.org> wrote:

> On 2/13/12 2:02 AM, ming.zym@gmail.com wrote:
>
>> that looks like you are in the issue of TS-1006:
>> https://issues.apache.org/**jira/browse/TS-1006<https://issues.apache.org/jira/browse/TS-1006>
>>
>
>
> Hmmm, so, I personally never see this unbounded growth myself. What is it
> that makes it grow out of bounds like this? It can't just be the RAM cache,
> is it? Is there a bug in how it calculates memory usage vs what is
> allocated for it?
>
> There is no surprise that memory isn't free, the freelist doesn't work
> like that (obviously). Meaning, if the system at some point needed 1GB of
> RAM, and it's now on the freelist, it would never go below 1GB of RAM
> usage. The questions are:
>
> 1) Is there a leak in the RAM cache, or some other bug that prevents it
> from limiting the memory usage as per the records.config settings?
>
> 2) Or is there a "leak" in the freelist, where objects that are put on the
> freelist are not reused, and instead, we allocate new ones?
>
>
> I think we have to try to understand why this is happening, what sort of
> bug it is, and how it happens. Blindly freeing things from the freelist
> doesn't seem right to me (it's goes against the design and purpose of it).
> Hence, understanding why the freelist is allowed to grow out of bounds is
> the first step.
>
> I do agree that it could be useful to have an (optional) feature where we
> periodically reduce the various freelists that we have, so that we can
> reclaim some memory from extreme "spikes" in usage. But that can't be the
> solution to this problem I don't think?
>
> -- Leif
>
>

Mime
View raw message