incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Theroux <mthero...@yahoo.com>
Subject Re: nodetool repair
Date Sun, 15 Jul 2012 16:14:42 GMT
So, if I have a 6 node cluster in the token ring, A->B->C->D-E->F, replication
factor 3, and I run repair (without -pr) on A, is the flow of information:

A synchronizes information it is responsible for with B and C (because B and C are replicas
of A).
A, as a replica of E and F, synchronizes E and F's information to itself.

If I re-ran repair of E, it would actually be redoing the work of synchronizing with A, but
will still be doing the worthwhile work of synchronizing with F.

Running repair with "-pr" will prevent this duplicate work, if you ran it on each node.  If
I ran repair on A with "-pr", A synchronizes its primary range with B and C, but does not
perform synchronization work with E and F.

Does this sound correct?

-Mike

On Jul 15, 2012, at 9:47 AM, Edward Capriolo wrote:

> Great job sleuthing.
> 
> Originally repair did not have a -pr. When you run the standard repair
> the node compares it's data with its neighbours and vice versa. They
> also send each other updates. Since you are supposed to repair every
> node < gc_grace submitting a full repair to each node would create
> duplicated work, since a repair on node A has an effect on node B and
> node C.
> 
> If you want to understand this some more you should run
> compactionstats and netstats across your cluster while a repair is
> going on, then you can see what effect the commands have on other
> nodes.
> 
> I will try to write up some documentation on it as well because -pr is
> a nice feature. Many may not even be expressly aware of it.
> 
> On Sat, Jul 14, 2012 at 2:00 PM, Michael Theroux <mtheroux2@yahoo.com> wrote:
>> Hello,
>> 
>> I'm looking at nodetool repair with the "-pr", vs. non "-pr" option.  Looking around,
I'm seeing a lot of conflicting information out there.  Almost universally, the recommendation
is to run nodetool repair with the "-pr" for any day-to-day maintenance.
>> 
>> This is my understanding of how it works.  I appreciate any corrections to my misinformation.
>> 
>> nodetool repair -pr
>> 
>> - This performs a repair on the "primary range" of the node.  The primary range is
essentially the part of the ring that the node is responsible for.  When this command is run,
synchronization of replicas will occur for the rows that this node is responsible for.  If
replicas are missing from that node's neighbors for those rows, they will be replicated.
>> 
>> nodetool repair
>> 
>> - This is where I see a lot of conflicting information.  I see a lot of answers in
which there is a suggestion that this command will perform a repair across the entire cluster.
 However, I don't believe this is true from my observations (and some of the items I read
seems to agree with this).  Instead, this command performs synchronization of your primary
range, but also for other ranges that this node maybe responsible for in a replica capacity.
 The way I'm thinking about it is that the -pr option causes repairs to push information from
its primary range to replicas.  Without -pr, nodetool replair does a push, and pull for its
neighbors that this node maybe a replica for.  This makes sense to me, as people recommend
running nodetool repair after a node has been down.  This is to allow the downed node to get
any missed information that should have been replicated to it while it was down.
>> 
>> I'm sure there lots of flaws to the above understanding as I'm cobbling it together.
 I appreciate the feedback,
>> 
>> -Mike


Mime
View raw message