Sure, I was testing using high traffic with about 6K - 7K req/sec reads and writes combined I added a node and ran repair, at this time the traffic was stopped and heap was 8G. I saw a lot of flushing and GC activity and finally it died saying out of memory. So I gave it more memory 12 G and started the nodes. This sped up the compactions and validations for around 12 hours and now I am back to the flushing and high GC activity at this point there was no traffic for more than 24 hours.

Again, thanks for the help !

On Wed, Oct 2, 2013 at 10:19 AM, cem <> wrote:
I think 512 is fine. Could you tell more about your traffic characteristics? 


On Wed, Oct 2, 2013 at 4:32 PM, srmore <> wrote:
I changed my index_interval from 128 to index_interval: 128 to 512, does it make sense to increase more than this ?

On Wed, Oct 2, 2013 at 9:30 AM, cem <> wrote:
Have a look to index_interval.


On Wed, Oct 2, 2013 at 2:25 PM, srmore <> wrote:
The version of Cassandra I am using is 1.0.11, we are migrating to 1.2.X though. We had tuned bloom filters (0.1) and AFAIK making it lower than this won't matter.

Thanks !

On Tue, Oct 1, 2013 at 11:54 PM, Mohit Anchlia <> wrote:
Which Cassandra version are you on? Essentially heap size is function of number of keys/metadata. In Cassandra 1.2 lot of the metadata like bloom filters were moved off heap.

On Tue, Oct 1, 2013 at 9:34 PM, srmore <> wrote:
Does anyone know what would roughly be the heap size for cassandra with 1TB of data ? We started with about 200 G and now on one of the nodes we are already on 1 TB. We were using 8G of heap and that served us well up until we reached 700 G where we started seeing failures and nodes flipping.

With 1 TB of data the node refuses to come back due to lack of memory. needless to say repairs and compactions takes a lot of time. We upped the heap from 8 G to 12 G and suddenly everything started moving rapidly i.e. the repair tasks and the compaction tasks. But soon (in about 9-10 hrs) we started seeing the same symptoms as we were seeing with 8 G.

So my question is how do I determine what is the optimal size of heap for data around 1 TB ?

Following are some of my JVM settings


Thanks !