cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: 0.7.9 RejectedExecutionException
Date Mon, 10 Oct 2011 20:18:49 GMT
The service keeps dieing at the same time every day and there is nothing in the app logs, it's
going to be something external.

Sorry but I'm not sure what the problem with the memory usage is. Is the server running out
of memory, or is it experiencing a lot of GC ? 

Cheers

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 11/10/2011, at 5:00 AM, Ashley Martens wrote:

> I have check both the output file and the system log, neither have errors in them. I
don't believe anything external is killing the process, I could be wrong but this node's setup
is the same as all my other nodes (including hardware) so it doesn't make much sense.
> 
> 
> jsvc.exec -user cassandra -home /usr/lib/jvm/java-6-openjdk/jre/bin/../ -pidfile /var/run/cassandra.pid
-errfile &1 -outfile /var/log/cassandra/output.log -cp /usr/share/cassandra/antlr-3.1.3.jar:/usr/share/cassandra/apache-cassandra-0.7.8.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/avro-1.4.0-fixes.jar:/usr/share/cassandra/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/commons-cli-1.1.jar:/usr/share/cassandra/commons-codec-1.2.jar:/usr/share/cassandra/commons-collections-3.2.1.jar:/usr/share/cassandra/commons-lang-2.4.jar:/usr/share/cassandra/concurrentlinkedhashmap-lru-1.1.jar:/usr/share/casandra/guava-r05.jar:/usr/share/cassandra/high-scale-lib.jar:/usr/share/cassandra/jackson-core-asl-1.4.0.jar:/usr/share/cassandra/jackson-mapper-asl-1.4.0.jar:/usr/share/cassandra/jetty-6.1.21.jar:/usr/share/cassandra/jetty-util-6.1.21.jar:/usr/share/cassandra/jline-0.9.94.jar:/usr/share/cassandra/json-simple-1.1.jar:/usr/share/cassandra/jug-2.0.0.jar:/usr/share/cassandra/libthrift-0.5.jar:/usr/share/cassandra/log4j-1.2.16.jar:/usr/share/cassandra/servlet-api-2.5-20081211.jar:/usr/share/cassandra/slf4j-api-1.6.1.jar:/usr/share/cassandra/slf4j-log4j12-1.6.1.jar:/usr/share/cassandra/snakeyaml-1.6.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar
-Dlog4j.configuration=log4j-server.properties -XX:HeapDumpPath=/var/lib/cassandra/java_1318260751.hprof
-XX:ErrorFile=/var/lib/casandra/hs_err_1318260751.log -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42
-Xms24196M -Xmx24196M -Xmn1600M -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true
-Dcom.sun.management.jmxremote.port=8080 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
org.apache.cassandra.thrift.CassandraDaemon
> 
> I have munin monitoring of JMX so when I talk about heap max then I'm referring to:
> 
> jmxObjectName java.lang:type=Memory
> jmxAttributeName HeapMemoryUsage
> jmxAttributeKey max
> 
> The other crazy thing is the heap used is no where close to heap max.
> 
> On Mon, Oct 10, 2011 at 12:40 AM, aaron morton <aaron@thelastpickle.com> wrote:
> Have you checked /var/log/cassandra/output.txt (the packaged install pipes std out/err
to there) or the system logs ? If there are no errors in the logs it may well be something
external killing it.
> 
> With regard to memory usage, it's hard for people to help unless you provide some numbers.
What do you mean by MAX heap ? Is this the max used heap size reported by JMX or the -Xmx
setting passed to the server ?
> 


Mime
View raw message