Which Cassandra version are you on? Essentially heap size is function of number of keys/metadata. In Cassandra 1.2 lot of the metadata like bloom filters were moved off heap.

On Tue, Oct 1, 2013 at 9:34 PM, srmore <comomore@gmail.com> wrote:
Does anyone know what would roughly be the heap size for cassandra with 1TB of data ? We started with about 200 G and now on one of the nodes we are already on 1 TB. We were using 8G of heap and that served us well up until we reached 700 G where we started seeing failures and nodes flipping.

With 1 TB of data the node refuses to come back due to lack of memory. needless to say repairs and compactions takes a lot of time. We upped the heap from 8 G to 12 G and suddenly everything started moving rapidly i.e. the repair tasks and the compaction tasks. But soon (in about 9-10 hrs) we started seeing the same symptoms as we were seeing with 8 G.

So my question is how do I determine what is the optimal size of heap for data around 1 TB ?

Following are some of my JVM settings


Thanks !