incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schuller <peter.schul...@infidyne.com>
Subject Re: Memory leak with Sun Java 1.6 ?
Date Tue, 14 Dec 2010 18:19:25 GMT
>> Memory-mapped files will account for both virtual and, to the extent
>> that they are resident in memory, to the resident size of the process.
>
>  This bears further investigation.  Would you consider a 3GB overhead
>  on a 4GB heap a possibility?  (From a position of some naivety, this
>  seems a bit extreme, though I'm continually surprised by Java.)

The point is that you might see 200 gb if you have 200 gb on files on
disk that are mapped, so when using mmap the resident size is
misleading.

>  This was my concern.  As mentioned, we're on 7GB machines,
>  4GB JVM heap, and an expectation that we'll get some gain
>  from the remainder being used sensibly by Linux as cache.

Data mmap():ed on file and resident is essentially LInux page cache -
it is just also mapped into the process.

>> Just to confirm, what does the free +/- buffers show if you run
>> 'free'? (I.e., middle line, under 'free' column)
>
>  Dang.  I thought I'd kept a cap of top and free on those boxes
>  before restarting Cassandra.  I'm confident that the item you're
>  talking about matches top's 'cached' entry - which at the time
>  was showing ~ 10MB.

free +/- buffers shows what the actual free memory is when discounting
the page cache. Since on most systems you'll always have page cache
filling up most memory (after some uptime), this is the only relevant
number to look at to answer the general question of 'how much memory
is left'.

It gets complicated when your software actually depends on the page
cache though.

>> A Java memory leak would likely indicate non-heap managed memory
>> (since I think it's unlikely that the JVM fails to limit the actual
>> heap size). The question is what....
>
>  I guess this comes back to my first question - (how) can a JVM with
>  a heap of 4GB get to 7.1GB of resident?  I understand a process can
>  blow out its virtual footprint, and despite a lack of swap on those
>  boxes, that bit isn't in itself a massive concern.

mmap():ed that's resident gets counted.

-- 
/ Peter Schuller

Mime
View raw message