cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Kumar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-8008) "Timed out waiting for timer thread" on large stress command
Date Fri, 26 Sep 2014 17:09:33 GMT
Shawn Kumar created CASSANDRA-8008:
--------------------------------------

             Summary: "Timed out waiting for timer thread" on large stress command
                 Key: CASSANDRA-8008
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8008
             Project: Cassandra
          Issue Type: Bug
          Components: Core
            Reporter: Shawn Kumar
         Attachments: node1.log, node2.log

I've been using cstar_perf to test cassandra with different gc's and came across this error
on one run which effectively stopped the test:

java.lang.RuntimeException: Timed out waiting for a timer thread - seems one got stuck at
org.apache.cassandra.stress.util.Timing.snap(Timing.java:83)

It looks similar to CASSANDRA-6943, but that should have fixed it, and I haven't been able
to consistently replicate this with other runs. This particular run was stress writing/reading
about 300M keys, and is an early attempt at carrying out a test of this size so perhaps it
only manifests with larger tests. 

The modifications from stock 2.1.0 were changes to heap size and usage of g1gc, as well as
using offheap_objects. I have attached thread dumps from the nodes in question, hopefully
they capture the broken state. I am continuing to test this, and will see if I can reproduce
this again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message