cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5351) Avoid repairing already-repaired data by default
Date Thu, 14 Nov 2013 18:57:30 GMT


Jonathan Ellis commented on CASSANDRA-5351:

I think everyone agrees that repaired data should have a leveling system exactly like current
LCS.  Freshly repaired sstables (streamed from other replicas) start in L0 and get leveled
from there.

The tricky part is, what do we do about the unrepaired data?  The two proposals are:

# Keep a separate unrepaired arena -- let's call it L0' to distinguish from the repaired L0
-- and perform STCS in it the way we do with an overflowing L0 currently.  Once repaired we
move sstables into L0 proper.
# Keep a complete extra set of levels for unrepaired data, L0', L1', ..., LN'.  Once repaired,
data from here will be dropped into repaired L0.

The downside to 1 is that you don't get full leveling benefits until it's repaired.  The downside
to 2 is that LCS already has super high write amplification properties (relative to STCS)
so doubling that is going to be even more painful.

Both downsides get mitigated by repairing more often.

Having written that out ... I'd lean towards the STCS option because it's so much simpler
to implement.

> Avoid repairing already-repaired data by default
> ------------------------------------------------
>                 Key: CASSANDRA-5351
>                 URL:
>             Project: Cassandra
>          Issue Type: Task
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Lyuben Todorov
>              Labels: repair
>             Fix For: 2.1
> Repair has always built its merkle tree from all the data in a columnfamily, which is
guaranteed to work but is inefficient.
> We can improve this by remembering which sstables have already been successfully repaired,
and only repairing sstables new since the last repair.  (This automatically makes CASSANDRA-3362
much less of a problem too.)
> The tricky part is, compaction will (if not taught otherwise) mix repaired data together
with non-repaired.  So we should segregate unrepaired sstables from the repaired ones.

This message was sent by Atlassian JIRA

View raw message