incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steffen Rusitschka <r...@megazebra.com>
Subject Compaction: pending tasks are stuck
Date Mon, 08 Jul 2013 08:45:01 GMT
Hi cassandra list,

we are having in issue in one of our Cassandra clusters where all 3 
nodes with RF 3 have 22 pending tasks (nodetool compactionstats) since 
mid of April that are never be executed. Even after a restart they show up.

It started with Cassandra 1.1.10 and on Friday we've migrated to 1.2.6 
and now we have the issue that flushes are no longer executed 
automatically - we have to do them every hour by hand (nodetool flush) 
in order to keep the nodes on an acceptable utilizatzion level. Without 
manual flush the CPU goes mad after a couple of hours on each instance.

We just added a forth node to the cluster and after the join it showed 0 
pending tasks. After decommissioning one of the old nodes the new node 
now shows 39 pending tasks that are apparently also stuck, even after 1h 
of idle time and a manual "nodetool compact":

$ nodetool compactionstats
pending tasks: 39
Active compaction remaining time :        n/a

I think the two issues are related since in the other clusters with 3 
and 10 nodes we don't have either of the issues.

Is there a way to find out which tasks are pending? Is there a way to 
cancel them?

Cheers
Steffen

-- 
Steffen Rusitschka
CTO
MegaZebra GmbH
Steinsdorfstraße 2
81538 München
Phone +49 89 80929577


rusi@megazebra.com
Challenge me at www.megazebra.com

MegaZebra GmbH
Geschäftsführer: Henning Kosmack, Christian Meister, Steffen Rusitschka
Sitz der Gesellschaft: München, HRB 177947

Mime
View raw message