Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 460FA11463 for ; Tue, 1 Jul 2014 15:44:54 +0000 (UTC) Received: (qmail 77845 invoked by uid 500); 1 Jul 2014 15:44:51 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 77808 invoked by uid 500); 1 Jul 2014 15:44:51 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 77797 invoked by uid 99); 1 Jul 2014 15:44:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Jul 2014 15:44:51 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_REMOTE_IMAGE X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ken.hancock@schange.com designates 74.125.82.181 as permitted sender) Received: from [74.125.82.181] (HELO mail-we0-f181.google.com) (74.125.82.181) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Jul 2014 15:44:49 +0000 Received: by mail-we0-f181.google.com with SMTP id q59so9843716wes.26 for ; Tue, 01 Jul 2014 08:44:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=9YyoerRg2TSnz5qiYdQl4UsKuWM22typVP+Z64apc78=; b=l089cJAGClfsFtrJ1QTZIXDgjHoJo2OjvvDb6ByOlPBfzQU7wl0KOOZ5hxX36I5pkA MM+86L4HXH27C3zCkAn4OjtAdNlfMRpcYsxzNLBOsoamJFjsKtmDKQ3Gn0oPkZzY5y6f BDMGfawR10cqRIrF2+LU44zuf6/8NN8cgNPDmBiazFZe+XKKJ8qw33hTaYBTjkLQxOOy rNGBhMlShnsP4uIMerGyGpD6COs9CmRPLRRq2XJB69QDTd2nz+oLpaiXktZ7jseWOCq+ QWCplfCvAkRdTo+KPHQKHrxJVzAJBv2zRfCr7zy8wowMSvRtYhLvKkgAX0Fi+30OxFhd wA+g== X-Gm-Message-State: ALoCoQm7Ks5R5OPLt037TqPwofE/qvOufsRnbg0pM8AUoW8nD+u4dqchGSavW/2fPKX1zAfcsfQF X-Received: by 10.180.81.1 with SMTP id v1mr5505089wix.10.1404229463298; Tue, 01 Jul 2014 08:44:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.190.230 with HTTP; Tue, 1 Jul 2014 08:44:03 -0700 (PDT) In-Reply-To: References: From: Ken Hancock Date: Tue, 1 Jul 2014 11:44:03 -0400 Message-ID: Subject: Re: nodetool repair -snapshot option? To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=f46d044306a057ed4604fd23a6db X-Virus-Checked: Checked by ClamAV on apache.org --f46d044306a057ed4604fd23a6db Content-Type: text/plain; charset=UTF-8 I also expanded on a script originally written by Matt Stump @ Datastax. The readme has the reasoning behind requiring sub-range repairs. https://github.com/hancockks/cassandra_range_repair On Mon, Jun 30, 2014 at 10:20 PM, Phil Burress wrote: > @Paulo, this is very cool! Thanks very much for the link! > > > On Mon, Jun 30, 2014 at 9:37 PM, Paulo Ricardo Motta Gomes < > paulo.motta@chaordicsystems.com> wrote: > >> If you find it useful, I created a tool where you input the node IP, >> keyspace, column family, and optionally the number of partitions (default: >> 32K), and it outputs the list of subranges for that node, CF, partition >> size: https://github.com/pauloricardomg/cassandra-list-subranges >> >> So you can basically iterate over the output of that and do subrange >> repair for each node and cf, maybe in parallel. :) >> >> >> On Mon, Jun 30, 2014 at 10:26 PM, Phil Burress >> wrote: >> >>> One last question. Any tips on scripting a subrange repair? >>> >>> >>> On Mon, Jun 30, 2014 at 7:12 PM, Phil Burress >>> wrote: >>> >>>> We are running repair -pr. We've tried subrange manually and that seems >>>> to work ok. I guess we'll go with that going forward. Thanks for all the >>>> info! >>>> >>>> >>>> On Mon, Jun 30, 2014 at 6:52 PM, Jaydeep Chovatia < >>>> chovatia.jaydeep@gmail.com> wrote: >>>> >>>>> Are you running full repair or on subset? If you are running full >>>>> repair then try running on sub-set of ranges which means less data to worry >>>>> during repair and that would help JAVA heap in general. You will have to do >>>>> multiple iterations to complete entire range but at-least it will work. >>>>> >>>>> -jaydeep >>>>> >>>>> >>>>> On Mon, Jun 30, 2014 at 3:22 PM, Robert Coli >>>>> wrote: >>>>> >>>>>> On Mon, Jun 30, 2014 at 3:08 PM, Yuki Morishita >>>>>> wrote: >>>>>> >>>>>>> Repair uses snapshot option by default since 2.0.2 (see NEWS.txt). >>>>>>> >>>>>> >>>>>> As a general meta comment, the process by which operationally >>>>>> important defaults change in Cassandra seems ad-hoc and sub-optimal. >>>>>> >>>>>> For to record, my view was that this change, which makes repair even >>>>>> slower than it previously was, was probably overly optimistic. >>>>>> >>>>>> It's also weird in that it changes default behavior which has been >>>>>> unchanged since the start of Cassandra time and is therefore probably >>>>>> automated against. Why was it so critically important to switch to snapshot >>>>>> repair that it needed to be shotgunned as a new default in 2.0.2? >>>>>> >>>>>> =Rob >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >> >> -- >> *Paulo Motta* >> >> Chaordic | *Platform* >> *www.chaordic.com.br * >> +55 48 3232.3200 >> > > -- *Ken Hancock *| System Architect, Advanced Advertising SeaChange International 50 Nagog Park Acton, Massachusetts 01720 ken.hancock@schange.com | www.schange.com | NASDAQ:SEAC Office: +1 (978) 889-3329 | [image: Google Talk:] ken.hancock@schange.com | [image: Skype:]hancockks | [image: Yahoo IM:]hancockks [image: LinkedIn] [image: SeaChange International] This e-mail and any attachments may contain information which is SeaChange International confidential. The information enclosed is intended only for the addressees herein and may not be copied or forwarded without permission from SeaChange International. --f46d044306a057ed4604fd23a6db Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I also expanded on a script originally written by Matt Stu= mp @ Datastax. The readme has the reasoning behind requiring sub-range repa= irs.

https://github.com/hancockks/cassandra_range_repair




On Mon, Jun 30, 2014 at 10:20 PM, Phil Burress <philburresseme@gma= il.com> wrote:
@Paulo, this is very cool! Thanks very= much for the link!


On Mon, Jun 3= 0, 2014 at 9:37 PM, Paulo Ricardo Motta Gomes <paulo.motta@c= haordicsystems.com> wrote:
If you find it useful,= I created a tool where you input the node IP, keyspace, column family, and= optionally the number of partitions (default: 32K), and it outputs the lis= t of subranges for that node, CF, partition size:=C2=A0https:= //github.com/pauloricardomg/cassandra-list-subranges

So you can basically iterate over the output of that and do subran= ge repair for each node and cf, maybe in parallel. :)


On Mon, Jun 3= 0, 2014 at 10:26 PM, Phil Burress <philburresseme@gmail.com>= wrote:
One last question. Any tips= on scripting a subrange repair?


On Mon, Jun 30, 2014 at 7:12 PM, Phil Bu= rress <philburresseme@gmail.com> wrote:
We are running repair -pr. = We've tried subrange manually and that seems to work ok. I guess we'= ;ll go with that going forward. Thanks for all the info!


On Mon, Jun 30, 2014 at 6:52 PM, Jaydeep Chovatia <chovatia.jayde= ep@gmail.com> wrote:
Are you running full repair or on subset? If you are runni= ng full repair then try running on sub-set of ranges which means less data = to worry during repair and that would help JAVA heap in general. You will h= ave to do multiple iterations to complete entire range but at-least it will= work.

-jaydeep


On Mon, Jun 30, 2014 at 3:22 = PM, Robert Coli <rcoli@eventbrite.com> wrote:
=
On Mon, Jun 30, 2014 at 3:08 PM, Yuki Moris= hita <mor.yuki@gmail.com> wrote:
Repair uses snapshot option by default since 2.= 0.2 (see NEWS.txt).

As a general meta comment, the = process by which operationally important defaults change in Cassandra seems= ad-hoc and sub-optimal.

For to record, my v= iew was that this change, which makes repair even slower than it previously= was, was probably overly optimistic.

It's also weird in that it changes default be= havior which has been unchanged since the start of Cassandra time and is th= erefore probably automated against. Why was it so critically important to s= witch to snapshot repair that it needed to be shotgunned as a new default i= n 2.0.2?

=3DRob
=C2=A0






<= /div>--
Paulo Motta





--
Ken Hancock=C2=A0| System Ar= chitect, Advanced Advertising=C2=A0
SeaChange International= =C2=A0
50 Nagog Park
Acton, Massachusetts 01720
ken.hancock@schange.com=C2=A0|=C2=A0www.sc= hange.com=C2=A0| NASDAQ:SEAC=C2=A0
Office: +1 (978) 889-3329=C2=A0|=C2=A03D"Google=C2=A0ken.hancock@schange.com=C2= =A0|=C2=A03D"Skype:"hancockks=C2=A0|=C2=A03D"Yahoohancockks		 3D"LinkedIn"

3D"SeaChange
This e-mail and any attachments may contain information which is SeaChange= International confidential. The information enclosed is intended only for = the addressees herein and may not be copied or forwarded without permission= from SeaChange International.
--f46d044306a057ed4604fd23a6db--