Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 47769DA0B for ; Tue, 28 Aug 2012 17:19:01 +0000 (UTC) Received: (qmail 22944 invoked by uid 500); 28 Aug 2012 17:18:58 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 22916 invoked by uid 500); 28 Aug 2012 17:18:58 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 22908 invoked by uid 99); 28 Aug 2012 17:18:58 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Aug 2012 17:18:58 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of coolmohitz@gmail.com designates 209.85.212.172 as permitted sender) Received: from [209.85.212.172] (HELO mail-wi0-f172.google.com) (209.85.212.172) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Aug 2012 17:18:52 +0000 Received: by wicr5 with SMTP id r5so6339760wic.7 for ; Tue, 28 Aug 2012 10:18:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=z2P5EBUZxPyaKBgcABUzEoltnRhiAn/vQqqKtdRWNC8=; b=JB82Kr7yQ77e1aEXsCkhP11BlTMFSW71of0ISoxW2RtFy6Hr4nQSlt8oXquDXitkRt aF+tAHWZdCeLxqG/kb6q4JDnJMaK1i5FtoquCJkzgm5pnKaMm43F2u61gw+4XFk2WPqC GzjbDl1fVthjO5iSyYZW6f8Dkjd8Bi/JbvSFowjsC06yF6X4AxqdWIyh6X3iFnk+u1DZ b2HzpfC9PekJLJRjjAPggA982+TCewjusJXtV3y4Yz/sWPcCiBZt8+6P+cwiSgK/rL01 qaqD/y3/SF/KAznTXnOTWWMzSnBxU2zTbHNF81xVoqVHqpmD3wiRvhP3lhg1cDNewfxo 9+dg== MIME-Version: 1.0 Received: by 10.180.100.133 with SMTP id ey5mr34329041wib.4.1346174312182; Tue, 28 Aug 2012 10:18:32 -0700 (PDT) Received: by 10.216.101.2 with HTTP; Tue, 28 Aug 2012 10:18:32 -0700 (PDT) In-Reply-To: References: <503C0853.2080505@globalrelay.net> Date: Tue, 28 Aug 2012 22:48:32 +0530 Message-ID: Subject: Re: Automating nodetool repair From: Mohit Agarwal To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=f46d0444ee27aef24f04c856a1fb --f46d0444ee27aef24f04c856a1fb Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Is there any reason why cassandra doesn't do nodetool repair out of the box at some fixed intervals? On Tue, Aug 28, 2012 at 9:08 PM, Aaron Turner wrote: > Funny you mention that... i just was hearing on #cassandra this > morning that it repairs the replica set by default. I was thinking of > repairing every 3rd node (RF=3D3), but running -pr seems "cleaner". > > Do you know if this (repairing a replica vs node) was introduced in 1.0 o= r > 1.1? > > On Tue, Aug 28, 2012 at 7:03 AM, Edward Capriolo > wrote: > > You can consider adding -pr. When iterating through all your hosts > > like this. -pr means primary range, and will do less duplicated work. > > > > On Mon, Aug 27, 2012 at 8:05 PM, Aaron Turner > wrote: > >> I use cron. On one box I just do: > >> > >> for n in node1 node2 node3 node4 ; do > >> nodetool -h $n repair > >> sleep 120 > >> done > >> > >> A lot easier then managing a bunch of individual crontabs IMHO > >> although I suppose I could of done it with puppet, but then you always > >> have to keep an eye out that your repairs don't overlap over time. > >> > >> On Mon, Aug 27, 2012 at 4:52 PM, Edward Sargisson > >> wrote: > >>> Hi all, > >>> So nodetool repair has to be run regularly on all nodes. Does anybody > have > >>> any interesting strategies or tools for doing this or is everybody ju= st > >>> setting up cron to do it? > >>> > >>> For example, one could write some Puppet code to splay the cron times > around > >>> so that only one should be running at once. > >>> Or, perhaps, a central orchestrator that is given some known quiet > time and > >>> works its way through the list, running nodetool repair one at a time > (using > >>> RPC?) until it runs out of time. > >>> > >>> Cheers, > >>> Edward > >>> -- > >>> > >>> Edward Sargisson > >>> > >>> senior java developer > >>> Global Relay > >>> > >>> edward.sargisson@globalrelay.net > >>> > >>> > >>> 866.484.6630 > >>> New York | Chicago | Vancouver | London (+44.0800.032.9829) | > Singapore > >>> (+65.3158.1301) > >>> > >>> Global Relay Archive supports email, instant messaging, BlackBerry, > >>> Bloomberg, Thomson Reuters, Pivot, YellowJacket, LinkedIn, Twitter, > Facebook > >>> and more. > >>> > >>> > >>> Ask about Global Relay Message =97 The Future of Collaboration in the > >>> Financial Services World > >>> > >>> > >>> All email sent to or from this address will be retained by Global > Relay=92s > >>> email archiving system. This message is intended only for the use of > the > >>> individual or entity to which it is addressed, and may contain > information > >>> that is privileged, confidential, and exempt from disclosure under > >>> applicable law. Global Relay will not be liable for any compliance o= r > >>> technical information provided herein. All trademarks are the > property of > >>> their respective owners. > >> > >> > >> > >> -- > >> Aaron Turner > >> http://synfin.net/ Twitter: @synfinatic > >> http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix > & Windows > >> Those who would give up essential Liberty, to purchase a little > temporary > >> Safety, deserve neither Liberty nor Safety. > >> -- Benjamin Franklin > >> "carpe diem quam minimum credula postero" > > > > -- > Aaron Turner > http://synfin.net/ Twitter: @synfinatic > http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & > Windows > Those who would give up essential Liberty, to purchase a little temporary > Safety, deserve neither Liberty nor Safety. > -- Benjamin Franklin > "carpe diem quam minimum credula postero" > --f46d0444ee27aef24f04c856a1fb Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Is there any reason why cassandra doesn't do nodetool repair out of the= box at some fixed intervals?=A0

On Tue, = Aug 28, 2012 at 9:08 PM, Aaron Turner <synfinatic@gmail.com> wrote:
Funny you mention that... i just was hearing= on #cassandra this
morning that it repairs the replica set by default. =A0I was thinking of repairing every 3rd node (RF=3D3), but running -pr seems "cleaner"= ;.

Do you know if this (repairing a replica vs node) was introduced in 1.0 or = 1.1?

On Tue, Aug 28, 2012 at 7:03 AM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
> You can consider adding -pr. When iterating through all your hosts
> like this. -pr means primary range, and will do less duplicated work.<= br> >
> On Mon, Aug 27, 2012 at 8:05 PM, Aaron Turner <synfinatic@gmail.com> wrote:
>> I use cron. =A0On one box I just do:
>>
>> for n in node1 node2 node3 node4 ; do
>> =A0 =A0nodetool -h $n repair
>> =A0 =A0sleep 120
>> done
>>
>> A lot easier then managing a bunch of individual crontabs IMHO
>> although I suppose I could of done it with puppet, but then you al= ways
>> have to keep an eye out that your repairs don't overlap over t= ime.
>>
>> On Mon, Aug 27, 2012 at 4:52 PM, Edward Sargisson
>> <edward.sar= gisson@globalrelay.net> wrote:
>>> Hi all,
>>> So nodetool repair has to be run regularly on all nodes. Does = anybody have
>>> any interesting strategies or tools for doing this or is every= body just
>>> setting up cron to do it?
>>>
>>> For example, one could write some Puppet code to splay the cro= n times around
>>> so that only one should be running at once.
>>> Or, perhaps, a central orchestrator that is given some known q= uiet time and
>>> works its way through the list, running nodetool repair one at= a time (using
>>> RPC?) until it runs out of time.
>>>
>>> Cheers,
>>> Edward
>>> --
>>>
>>> Edward Sargisson
>>>
>>> senior java developer
>>> Global Relay
>>>
>>> edward.sar= gisson@globalrelay.net
>>>
>>>
>>> 866.484.66= 30
>>> New York | Chicago | Vancouver =A0| =A0London =A0(+44.0800.032.9829) = =A0| =A0Singapore
>>> (+65.= 3158.1301)
>>>
>>> Global Relay Archive supports email, instant messaging, BlackB= erry,
>>> Bloomberg, Thomson Reuters, Pivot, YellowJacket, LinkedIn, Twi= tter, Facebook
>>> and more.
>>>
>>>
>>> Ask about Global Relay Message =97 The Future of Collaboration= in the
>>> Financial Services World
>>>
>>>
>>> All email sent to or from this address will be retained by Glo= bal Relay=92s
>>> email archiving system. This message is intended only for the = use of the
>>> individual or entity to which it is addressed, and may contain= information
>>> that is privileged, confidential, and exempt from disclosure u= nder
>>> applicable law. =A0Global Relay will not be liable for any com= pliance or
>>> technical information provided herein. =A0All trademarks are t= he property of
>>> their respective owners.
>>
>>
>>
>> --
>> Aaron Turner
>> http://synfin.net= / =A0 =A0 =A0 =A0 Twitter: @synfinatic
>> http://= tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Wi= ndows
>> Those who would give up essential Liberty, to purchase a little te= mporary
>> Safety, deserve neither Liberty nor Safety.
>> =A0 =A0 -- Benjamin Franklin
>> "carpe diem quam minimum credula postero"



--
Aaron Turner
http://synfin.net/ =A0= =A0 =A0 =A0 Twitter: @synfinatic
http://tcpreplay= .synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.
=A0 =A0 -- Benjamin Franklin
"carpe diem quam minimum credula postero"

--f46d0444ee27aef24f04c856a1fb--