cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aoi Kadoya <cadyan....@gmail.com>
Subject CPU high load
Date Tue, 12 Jul 2016 22:23:53 GMT
Hi,

I am running 6 nodes vnode cluster with DSE 4.8.1, and since few weeks
ago, all of the cluster nodes are hitting avg. 15-20 cpu load.
These nodes are running on VMs(VMware vSphere) that have 8vcpu
(1core/socket)-16 vRAM.(JVM options : -Xms8G -Xmx8G -Xmn800M)

At first I thought this is because of CPU iowait, however, iowait is
constantly low(in fact it's 0 almost all time time), CPU steal time is
also 0%.

When I took a thread dump, I found some of "SharedPool-Worker" threads
are consuming CPU and those threads seem to be waiting for something
so I assume this is the cause of cpu load.

"SharedPool-Worker-1" #240 daemon prio=5 os_prio=0
tid=0x00007fabf459e000 nid=0x39b3 waiting on condition
[0x00007faad7f02000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:85)
at java.lang.Thread.run(Thread.java:745)

Thread dump looks like this, but I am not sure what is this
sharedpool-worker waiting for.
Would you please help me with the further trouble shooting?
I am also reading the thread posted by Yuan as the situation is very
similar to mine but I didn't get any blocked, dropped or pending count
in my tpstat result.

Thanks,
Aoi

Mime
View raw message