cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5972) Reduce the amount of data to be transferred during repair
Date Tue, 03 Sep 2013 12:10:56 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756544#comment-13756544
] 

Sylvain Lebresne commented on CASSANDRA-5972:
---------------------------------------------

Right. But that does assume a bunch of things that repair currently doesn't do like recomputing
hashes post streaming, exchanging merkle tree update and a lot more coordination (we pretty
much would need all nodes to have all merkle trees, while today repair uses a single node
to do the whole synchronization).

So please don't get me wrong: I'm in no way saying that repair cannot be made to transfer
less data (nor that your suggestion wouldn't allow that). I'm saying that the repair code
(the MerkleTree class in particular) are a lot uglier than then could be, making changes like
the one this ticket suggest (or CASSANDRA-3200 for that matter) harder to do that it sounds
on paper. And that to make your suggestion work in practice is a little more complicated that
what your initial description made it sound.

Now, I do abolutely believe that we should fix that excessive transfer at some point, but
I also think that once we invest the energy to refactor repair to fix that then CASSANDRA-3200
is imo a better solution since it minimizes the number of transfer done, while I don't think
the solution here does (at least not always).

                
> Reduce the amount of data to be transferred during repair
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-5972
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5972
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jacek Lewandowski
>            Priority: Minor
>
> Currently, when a validator finds a token range different in n replicas, data streams
are initiated simultaneously between each possible pair of these n nodes, in both directions.
It yields n*(n-1) data stream in total. 
> It can be done in a sequence - Replica[1] -> R[2], R[2] -> R[3], ... , R[n-1] ->
R[n]. After this process, the data in R[n] are up to date. Then, we continue: R[n] -> R[1],
R[1] -> R[2], ... , R[n-2] -> R[n-1]. The active repair is done after 2*(n-1) data transfers
performed sequentially in 2*(n-1) steps.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message