cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Repair failure under 0.8.6
Date Mon, 05 Dec 2011 00:52:03 GMT
You can say the min compaction threshold to 2 and the max Compaction
Threshold to 3. If you have enough disk space for a few minor compaction
this should free up some disk space.

On Sun, Dec 4, 2011 at 7:17 PM, Peter Schuller
<peter.schuller@infidyne.com>wrote:

> > As a side effect of the failed repair (so it seems) the disk usage on the
> > affected node prevents compaction from working. It still works on
> > the remaining nodes (we have 3 total).
> > Is there a way to scrub the extraneous data?
>
> This is one of the reasons why killing an in-process repair is a bad thing
> :(
>
> If you do not have enough disk space for any kind of compaction to
> work, then no, unfortunately there is no easy way to get rid of the
> data.
>
> You can go to extra trouble such as moving the entire node to some
> other machine (e.g. firewalled from the cluster) with more disk and
> run compaction there and then "move it back", but that is kind of
> painful to do. Another option is to decommission the node and replace
> it. However, be aware that (1) that leaves the ring with less capacity
> for a while, and (2) when you decommission, the data you stream from
> that node to others would be artificially inflated due to the repair
> so there is some risk of "infecting" the other nodes with a large data
> set.
>
> I should mention that if you have no traffic running against the
> cluster, one way is to just remove all the data and then run repair
> afterwards. But that implies that you're trusting that (1) no reads
> are going to the cluster (else you might serve reads based on missing
> data) and (2) that you are comfortable with loss of the data on the
> node. (2) might be okay if you're e.g. writing at QUORUM at all times
> and have RF >= 3 (basically, this is as if the node would have been
> lost due to hardware breakage).
>
> A faster way to reconstruct the node would be to delete the data from
> your keyspaces (except the system keyspace), start the node (now
> missing data), and run 'nodetool rebuild' after
> https://issues.apache.org/jira/browse/CASSANDRA-3483 is done. The
> patch attached to that ticket should work for 0.8.6 I suspect (but no
> guarantees). This also assumes you have no reads running against the
> cluster.
>
> --
> / Peter Schuller (@scode, http://worldmodscode.wordpress.com)
>

Mime
View raw message