We use Cassandra version 0.6.11. Our cache size is very small.  11 out of 12 servers have used heap size less than 500MB of 3GB allocated. Just one server that had memory usage run out of control.

The issue is isolated.  It turn out that one CF has a row with compacted row size of 50MB.  And this row happens to be very active row that get updated every few seconds.   By deleting this row, heap memory usage is consistently low across servers on the cluster.

It came to my surprise how expensive it is to have this row of 50MB compacted size being updated every few seconds.  Does anyone have any insight as to why the row mentioned is so expensive to have?

Thanks!

Huy

On Wed, Feb 9, 2011 at 2:34 PM, Robert Coli <rcoli@digg.com> wrote:
On Wed, Feb 9, 2011 at 11:04 AM, Huy Le <huyle@springpartners.com> wrote:
> Memory usage grows overtime.

It is relatively typical for caches to exert memory pressure over time
as they fill. What are your cache settings, for how many
columnfamilies, and with what sized memtables? What version of
Cassandra?

=Rob



--
Huy Le
Spring Partners, Inc.
http://springpadit.com