incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jedd Rashbrooke <jedd.rashbro...@imagini.net>
Subject Re: Memory leak with Sun Java 1.6 ?
Date Tue, 14 Dec 2010 10:43:16 GMT
 Peter, Jonathon - thank you for your replies.

 I should probably have repeated myself in the body, but as I
 mentioned in the subject line, we're running Sun Java 1.6.


On 10 December 2010 18:37, Peter Schuller <peter.schuller@infidyne.com> wrote:
> Memory-mapped files will account for both virtual and, to the extent
> that they are resident in memory, to the resident size of the process.

 This bears further investigation.  Would you consider a 3GB overhead
 on a 4GB heap a possibility?  (From a position of some naivety, this
 seems a bit extreme, though I'm continually surprised by Java.)


> However, your graph:
>
>>  I've shared a Zabbix graph of system memory at:
>>
>>     http://www.imagebam.com/image/3b4213110283969
>
> Certainly indicates that it is not the explanation since you should be
> seeing cached occupy the remainder of memory above heap size. In
> addition the allocation failures from jmap indicates memory is truly
> short.

 This was my concern.  As mentioned, we're on 7GB machines,
 4GB JVM heap, and an expectation that we'll get some gain
 from the remainder being used sensibly by Linux as cache.

> Just to confirm, what does the free +/- buffers show if you run
> 'free'? (I.e., middle line, under 'free' column)

 Dang.  I thought I'd kept a cap of top and free on those boxes
 before restarting Cassandra.  I'm confident that the item you're
 talking about matches top's 'cached' entry - which at the time
 was showing ~ 10MB.

> A Java memory leak would likely indicate non-heap managed memory
> (since I think it's unlikely that the JVM fails to limit the actual
> heap size). The question is what....

 I guess this comes back to my first question - (how) can a JVM with
 a heap of 4GB get to 7.1GB of resident?  I understand a process can
 blow out its virtual footprint, and despite a lack of swap on those
 boxes, that bit isn't in itself a massive concern.



On 12 December 2010 16:21, Jonathan Ellis <jbellis@gmail.com> wrote:
> http://www.riptano.com/docs/0.6/troubleshooting/index#nodes-are-dying-with-oom-errors

 Jonathon - to clarify - this seems to be happening *to* Java, not
 within the JVM.


 I posted mostly as a heads up for others using similar profiles (4GB
 heap on ~8GB boxes) to keep an eye out for.  I expect a few people,
 particularly if they're on Amazon EC2, are running this type of setup.

 On the other hand, mum always said I was unique.  ;)

 cheers,
 Jedd.

Mime
View raw message