Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
Sender: scode@scode.org
In-Reply-To: <AANLkTi=VRJVPvoe+stqucE+vDzmA4ioEA2VCqDSi6Kak@mail.gmail.com>
References: <AANLkTik0uC_vX=8obMdnVcetia15qQa6AtnHoaaaq2yw@mail.gmail.com>
	<AANLkTi=u5gNFhc9mmm8mD3K0d7oqGY74wAmV5nE4PhcT@mail.gmail.com>
	<AANLkTi=VRJVPvoe+stqucE+vDzmA4ioEA2VCqDSi6Kak@mail.gmail.com>
Date: Tue, 14 Dec 2010 19:19:25 +0100
Message-ID: <AANLkTinTGgMmVfv=EncnsJiE=NCehBM+_XABzOQ8+S21@mail.gmail.com>
Subject: Re: Memory leak with Sun Java 1.6 ?
From: Peter Schuller <peter.schuller@infidyne.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

>> Memory-mapped files will account for both virtual and, to the extent
>> that they are resident in memory, to the resident size of the process.
>
> =C2=A0This bears further investigation. =C2=A0Would you consider a 3GB ov=
erhead
> =C2=A0on a 4GB heap a possibility? =C2=A0(From a position of some naivety=
, this
> =C2=A0seems a bit extreme, though I'm continually surprised by Java.)

The point is that you might see 200 gb if you have 200 gb on files on
disk that are mapped, so when using mmap the resident size is
misleading.

> =C2=A0This was my concern. =C2=A0As mentioned, we're on 7GB machines,
> =C2=A04GB JVM heap, and an expectation that we'll get some gain
> =C2=A0from the remainder being used sensibly by Linux as cache.

Data mmap():ed on file and resident is essentially LInux page cache -
it is just also mapped into the process.

>> Just to confirm, what does the free +/- buffers show if you run
>> 'free'? (I.e., middle line, under 'free' column)
>
> =C2=A0Dang. =C2=A0I thought I'd kept a cap of top and free on those boxes
> =C2=A0before restarting Cassandra. =C2=A0I'm confident that the item you'=
re
> =C2=A0talking about matches top's 'cached' entry - which at the time
> =C2=A0was showing ~ 10MB.

free +/- buffers shows what the actual free memory is when discounting
the page cache. Since on most systems you'll always have page cache
filling up most memory (after some uptime), this is the only relevant
number to look at to answer the general question of 'how much memory
is left'.

It gets complicated when your software actually depends on the page
cache though.

>> A Java memory leak would likely indicate non-heap managed memory
>> (since I think it's unlikely that the JVM fails to limit the actual
>> heap size). The question is what....
>
> =C2=A0I guess this comes back to my first question - (how) can a JVM with
> =C2=A0a heap of 4GB get to 7.1GB of resident? =C2=A0I understand a proces=
s can
> =C2=A0blow out its virtual footprint, and despite a lack of swap on those
> =C2=A0boxes, that bit isn't in itself a massive concern.

mmap():ed that's resident gets counted.

--=20
/ Peter Schuller