incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Heap usage: Some clarification
Date Fri, 04 Feb 2011 16:08:34 GMT
by default cassandra will use mmap data access on 64 bit environments.

meaning, we don't _allocate_ anything off heap, but mmapped data
counts towards resident memory as reported by top.  the OS manages
paging it in on demand.

On Fri, Feb 4, 2011 at 5:06 AM, Nick Telford <nick.telford@gmail.com> wrote:
> Hi guys,
>
> Users frequently ask questions regarding memory tuning and heap usage; one
> of the more common being: "why does top report resident memory usage larger
> than -Xmx?".
>
> I'd like to get some clarification on the usage patterns in Cassandra so I
> write something up in the Wiki and generally answer questions better.
>
> I understand that the -Xm(s|x) flags only affect the JVM managed heap, and
> that memory is also allocated for the stack (per-thread), program code and
> native heap. Of these, I can only see the native heap being a major
> contributor to memory usage. Would I be right in saying that this is due to
> Cassandra's liberal use of NIO direct ByteBuffers? I couldn't find a whole
> lot of information on JVM memory usage, but what I did find[1] suggests that
> these are allocated outside of the native heap (with only a pointer object
> in the JVM heap).
>
> If that's correct, what specifically are direct ByteBuffers used for and
> when were they introduced? I seem to recall a lot of porting to ByteBuffers
> in 0.7, yet our production 0.6 cluster exhibits a high non-JVM heap usage
> that indicates otherwise.
>
> If I'm way off the mark, please could you guys point me in the right
> direction.
>
>
> [1]: http://www.ibm.com/developerworks/java/library/j-nativememory-linux/
>
> Cheers,
>
> Nick Telford
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Mime
View raw message