cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Molinaro <antho...@alumni.caltech.edu>
Subject Recovery from botched compaction
Date Sat, 10 Apr 2010 19:24:38 GMT
Hi,

  This is sort of a pre-emptive question as the compaction I'm doing hasn't
failed yet but I expect it to any time now.  I have a cluster which has been
storing user profile data for a client.  Recently I've had to go back and
reload all the data again.  I wasn't watching diskspace, and on one of the
nodes it went above 50% (which I recall was bad), to somewhere around 70%.
I expect to most back with a compaction (as most of the data was the same
so a compaction should remove old copies), and went ahead and started one
with nodeprobe compact (using 0.5.0 on this cluster).  However, I do see
that the disk usage is growing (it's at 91% now).

So when the disk fills up and this compaction crashes what can I do?
I assume get a bigger disk, shut down the node, move the data and
restart will work, but do I have other options?
Which files can I ignore (ie, can I not move any of the *-tmp-* files)?
Will my system be in a corrupt state?

This machine is one in a set of 6, and since I didn't choose tokens
initially, they are very lopsided (ie, some use 20% of their disk, others
60-70%).  If I were to start moving tokens around would the machines short
of space be able to anti-compact without filling up?  or does anti-compaction
like compaction require 2x disk space?

Thanks,

-Anthony

-- 
------------------------------------------------------------------------
Anthony Molinaro                           <anthonym@alumni.caltech.edu>

Mime
View raw message