incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schuller <peter.schul...@infidyne.com>
Subject Re: Fatal error from a cassandra node
Date Wed, 30 Mar 2011 18:41:48 GMT
>            I have 6 node cassandra cluster all are setup with same
> configuration    I am getting fatal exceptions in one of the nodes
> ERROR [Thread-604] 2011-03-29 20:19:13,218 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[Thread-604,5,main]
> java.lang.OutOfMemoryError: Java heap space
> ERROR [Thread-607] 2011-03-29 19:47:29,272 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[Thread-607,5,main]
> java.lang.OutOfMemoryError: Java heap space
> ERROR [Thread-605] 2011-03-29 19:38:09,081 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[Thread-605,5,main]
> java.lang.OutOfMemoryError: Java heap space
> ERROR [MutationStage:2] 2011-03-29 19:37:16,659
> DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor
> java.lang.OutOfMemoryError: Java heap space
> ERROR [GossipStage:1] 2011-03-29 20:27:29,898 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[GossipStage:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
>
> All all the nodes have 32 G of ram.
>
> Everytime I try to restart the failed node I get the above errors.

Unless something is outright wrong, it sounds like you need to
increase your JVM heap size in cassandra-env.sh. That you're getting
it on start-up sounds consistent with commit log reply filling the
heap in the form of memtables that are sized too big for your heap.

There's a wiki page somewhere that describes the overall rule of thumb
for heap sizing, but I can't find it right now.

-- 
/ Peter Schuller

Mime
View raw message