cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Eriksson (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5351) Avoid repairing already-repaired data by default
Date Tue, 07 Jan 2014 06:42:56 GMT


Marcus Eriksson commented on CASSANDRA-5351:

this is what i have so far;

* CompactionTask 
  - should be kept generic, move the compaction strategy specific stuff into the classes (we
still want to support users writing their own strategies)
* CompactionManager
  - Calculates merkle trees over all data, not just the new unrepaired stuff.
  - Marks all data as compacting (should only mark the unrepaired sstables as compacting).
  - Since we need to keep the involved sstables from getting compacted during the whole repair,
we might have to add a timeout - if repair hangs, we would essentially stop compacting the
new data. We might also need a "fall back to repair-everything if the amount of new data is
more than X"-config variable to avoid blocking compactions
  - Don't use System.nanoTime() as repairedAt time - nanoTime should only be used to measure
time between two events in the same jvm. Use System.currentTimeMillis() instead.

> Avoid repairing already-repaired data by default
> ------------------------------------------------
>                 Key: CASSANDRA-5351
>                 URL:
>             Project: Cassandra
>          Issue Type: Task
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Lyuben Todorov
>              Labels: repair
>             Fix For: 2.1
>         Attachments: 5351_node1.log, 5351_node2.log, 5351_node3.log, 5351_nodetool.log
> Repair has always built its merkle tree from all the data in a columnfamily, which is
guaranteed to work but is inefficient.
> We can improve this by remembering which sstables have already been successfully repaired,
and only repairing sstables new since the last repair.  (This automatically makes CASSANDRA-3362
much less of a problem too.)
> The tricky part is, compaction will (if not taught otherwise) mix repaired data together
with non-repaired.  So we should segregate unrepaired sstables from the repaired ones.

This message was sent by Atlassian JIRA

View raw message