cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Di Spaltro <dan.dispal...@gmail.com>
Subject Stalled Bootstrapping Process
Date Thu, 01 Apr 2010 16:26:18 GMT
So we are adding another node to the cluster with the latest 0.6 branch
(RC1).  It seems to be hung in some limbo state.

Before bootstrapping our cluster had 50-60GB spread fairly evenly across 4
machines, with RF=3.   One machine had more load than the others, and sure
enough bootstrapping selected that node.   That is the red machine.  The
light blue machine is the new machine.

I have attached a graph to illustrate when the bootstrap process started.

In jconsole the streamingservice status was "performing anticompaction..."
for over 18-24 hrs.  It is currently in "nothing is happening".   It did
have 1 active STREAM-STAGE task, but the machine had to be rebooted for
something unrelated to cassandra. Now the light blue machine appears to be
getting data, but its growing at virtually the same rate as the other
machines which makes me think it is part of the cluster and not actually
streaming data from the machine its supposed to.

Any other ideas on how to debug?


-- 
Dan Di Spaltro

Mime
View raw message