incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "B. Todd Burruss" <bburr...@real.com>
Subject cassandra not responding
Date Tue, 16 Mar 2010 16:39:06 GMT
i have a cassandra node that has stopped responding to nodetool tpstats 
and not processing requests.  memory is maxed on the JVM at 44gig.  the 
last i remember seeing from tpstats was over 150 hinted handoffs 
pending.  there hasn't been an OOM in the log yet, so not sure.  JMX 
isn't responding fast enough either to give stats.

since the JVM appears to be maxed on RAM i imagine GC is preventing 
everything from working properly, but not sure why maxed.  i have seen 
memory max out from time to time during bootstrapping, moving tokens, 
etc but can't get any stats when this happens because JVM isn't responding.

just want to throw this out to the list and see if anyone else has seen 
issues with memory creeping up and eventually no more reponse from node.

any other ideas on how to troubleshoot?  i have tried kill -3 <java_pid> 
in the past but don't know where cassandra writes the console out.  i'll 
look at scripts.

thx

Mime
View raw message