incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schuller <>
Subject Re: Dazed and confused with Cassandra on EC2 ...
Date Thu, 07 Oct 2010 19:49:54 GMT
>  There's some words on the 'Net that - the recent pages on
>  Riptano's site in fact - that strongly encourage scaling left
>  and right, rather than beefing up the boxes - and certainly
>  we're seeing far less bother from GC using a much smaller
>  heap - previously we'd been going up to 16GB, or even
>  higher.  This is based on my previous positive experiences
>  of getting better performance from memory hog apps (eg.
>  Java) by giving them more memory.  In any case, it seems
>  that using large amounts of memory on EC2 is just asking
>  for trouble.

Keep in mind that while GC tends to be more efficient with larger heap
sizes, that does not always translate into better overall performance
when other things have to be considered. In particular, in the case of
Cassandra, if you "waste" 10-15 gigs of RAM on the JVM heap for a
Cassandra instances which could live with e.g. 1 GB, you're actively
taking away those 10-15 gigs of RAM from the operating system to use
for the buffer cache. Particularly if you're I/O bound on reads then,
this could have very detrimental effects (assuming the data set is
sufficiently small and locality is such that 15 GB of extra buffer
cache makes a difference; usually, but not always, this is the case).

So with Cassandra, in the general case, you definitely want to keep
hour heap size reasonable in relation to the actual live set (amount
of actually reachable data), rather than just cranking it up as much
as possible.

(The main issue here is also keeping it high enough to not OOM, given
that exact memory demands are hard to predict; it would be absolutely
great if the JVM was better at maintaining a reasonable heap size to
live set size ratio so that much less tweaking of heap sizes was
necessary, but this is not the case.)

/ Peter Schuller

View raw message