incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Theroux <mthero...@yahoo.com>
Subject Re: -pr vs. no -pr
Date Fri, 01 Mar 2013 03:05:26 GMT
The way I've always thought about it is that -pr will make sure the information that specific
node originates is consistent with its replicas.

So, we know that a node is responsible for a specific token range, and the next nodes in the
ring will hold its replicas.  The -pr will make sure that a specific node's information is
consistent to its replicas, but will not make sure a specific node has all the replicated
information it can get from nodes previous to itself in the ring.

Without the -pr option, not only will the current node make sure its information and its replica's
information is consistent, but it will also make sure that all the information that it is
a replica for, is consistent.  

If you run regular repairs on all the nodes in your cluster, then -pr is sufficient.  Every
node will run repair, and make sure its information is consistent with its replicas, eventually
creating a fully consistent cluster.  This is a quicker process, and will have less impact
on your operations by essentially spreading out the pain.  

For instance, we run a 12 node cluster.  We run "nodetool repair -pr" on nodes that are opposite
to each other, 4 nodes a day (2 nodes in the morning, 2 nodes in the evening).  With a grace
period of 10 days, this allows us to run repairs twice a week on a specific node, and to occasionally
skip repairs on specific nodes once a week.  

In this case, without -pr, a lot of extra work would be done.  In fact, with an RF of 3 (in
our case), the time per repair would increase many fold.

Another way to thing about it... although likely not 100% technically correct..

A repair -pr will cause a push of a node's information to its replicas.  Without the -pr,
it will cause a push, and it will cause nodes it is a replica for to push their information
as well.

-Mike

On Feb 28, 2013, at 9:39 PM, Hiller, Dean wrote:

> Isn't there more to it than that.  You really have nodes responsible for
> token ranges like so(using describe ring)
> 
> What we see is this from our describe ringŠ(1 to 6 are token ranges while
> A to F are servers)Š.
> A - 1, 2, 3
> B - 2, 3, 4
> C - 3, 4, 5
> D - 4, 5, 6
> E - 5, 6, 1
> F - 6, 1, 2
> 
> With -pr, only token range 1 is repaired I think, right?  2 and 3 are only
> repaired without the -pr option?  This means if I have a node that I just
> joined the cluster, I should "not" be using -pr as 2 and 3 on node A will
> not be up to date.  Using -pr is nice if I am going to repair every single
> node and is nice for the cron job that has to happen before
> gc_grace_seconds.  Am I wrong here?  Ie. -pr is really only good for use
> in the cron job as it would miss 2 and 3 above.  I could run the cron on
> just two servers but then my nodes are different which can be a hassle.
> 
> Please verify that is what you believe is what happens as well?
> 
> Thanks,
> Dean
> 
> On 2/28/13 5:58 PM, "Takenori Sato(Cloudian)" <tsato@cloudian.com> wrote:
> 
>> Hi,
>> 
>> Please note that I confirmed on v1.0.7.
>> 
>>> I mean a repair involves all three nodes and pushes and pulls data,
>> right?
>> 
>> Yes, but that's how -pr works. A repair without -pr does more.
>> 
>> For example, suppose you have a ring with RF=3 like this.
>> 
>> A - B - C - D - E - F
>> 
>> Then, a repair on A without -pr does for 3 ranges as follows:
>> [A, B, C]
>> [E, F, A]
>> [F, A, B]
>> 
>> Among them, the first one, [A, B, C] is the primary range of A.
>> 
>> So, with -pr, a repair runs only for:
>> [A, B, C]
>> 
>>> I could run nodetool repair on just 2 nodes(RF=3) instead of using
>> nodetool repair ­pr???
>> 
>> Yes.
>> 
>> You need to run two repairs on A and D.
>> 
>>> What is the advantage of ­pr then?
>> 
>> Whenever you want to minimize rapair impacts.
>> 
>> For example, suppose you got one node down for a while, and bring it
>> back to the cluster.
>> 
>> You need to run rapair without affecting the entire cluster. Then, -pr
>> is the option.
>> 
>> Thanks,
>> Takenori
>> 
>> (2013/03/01 7:39), Hiller, Dean wrote:
>>> Isn't it true if I have 6 nodes, I could run nodetool repair on just 2
>>> nodes(RF=3) instead of using nodetool repair ­pr???
>>> 
>>> What is the advantage of ­pr then?
>>> 
>>> I mean a repair involves all three nodes and pushes and pulls data,
>>> right?
>>> 
>>> Thanks,
>>> Dean
>> 
> 


Mime
View raw message