Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4F10411996 for ; Tue, 1 Jul 2014 01:38:46 +0000 (UTC) Received: (qmail 79368 invoked by uid 500); 1 Jul 2014 01:38:43 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 79327 invoked by uid 500); 1 Jul 2014 01:38:43 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 79312 invoked by uid 99); 1 Jul 2014 01:38:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Jul 2014 01:38:43 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of paulo.motta@chaordicsystems.com designates 209.85.160.173 as permitted sender) Received: from [209.85.160.173] (HELO mail-yk0-f173.google.com) (209.85.160.173) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Jul 2014 01:38:41 +0000 Received: by mail-yk0-f173.google.com with SMTP id 142so5223213ykq.32 for ; Mon, 30 Jun 2014 18:38:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=RAAOOOzrfqVFuBMHpf76P9ZI4XZxl5usErHRvoLNssc=; b=hIysWD+ovlbGFhlqZFZ3mwLJDLlVDFe4OexAw9FSGXFNnF0rQ3LTD+zFW6mqmXu4Dx GXuHORSL3aZDxqCdbX/P0RbzQsLIs5/kX2R2vWJbCj2xnhM6z7P+AO7vnHu66kdttOA4 w1wMNg3oNBFR3RMkXVr0C1VkH5noytMjUlNyxgF+Jtoqzsrdn2YOZDWpjltEghQ544Vq wlAlqQiurv1G4PcLcj33UfGlB7Fi401O3YQnmAhKLov0W3xNIIR5/lzZZe3UWF/bo9Ok sI/OPZ5KsZwWNp9kFvQ2D6KmdbYlh+28wk2OIj/G62/0bMO/HqdoDYTzZCz/6pcCiIDO NUfQ== X-Gm-Message-State: ALoCoQkxwRehJDv8hoKx/mYFiYb5bDT4C8UuYo8gL4BkgFY1FtJwH7Y4MeImpYnLRUpgTamMaA44 X-Received: by 10.236.60.134 with SMTP id u6mr9202858yhc.137.1404178696837; Mon, 30 Jun 2014 18:38:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.170.216.136 with HTTP; Mon, 30 Jun 2014 18:37:56 -0700 (PDT) In-Reply-To: References: From: Paulo Ricardo Motta Gomes Date: Mon, 30 Jun 2014 22:37:56 -0300 Message-ID: Subject: Re: nodetool repair -snapshot option? To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a11c1edec6d320a04fd17d4b7 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c1edec6d320a04fd17d4b7 Content-Type: text/plain; charset=UTF-8 If you find it useful, I created a tool where you input the node IP, keyspace, column family, and optionally the number of partitions (default: 32K), and it outputs the list of subranges for that node, CF, partition size: https://github.com/pauloricardomg/cassandra-list-subranges So you can basically iterate over the output of that and do subrange repair for each node and cf, maybe in parallel. :) On Mon, Jun 30, 2014 at 10:26 PM, Phil Burress wrote: > One last question. Any tips on scripting a subrange repair? > > > On Mon, Jun 30, 2014 at 7:12 PM, Phil Burress > wrote: > >> We are running repair -pr. We've tried subrange manually and that seems >> to work ok. I guess we'll go with that going forward. Thanks for all the >> info! >> >> >> On Mon, Jun 30, 2014 at 6:52 PM, Jaydeep Chovatia < >> chovatia.jaydeep@gmail.com> wrote: >> >>> Are you running full repair or on subset? If you are running full repair >>> then try running on sub-set of ranges which means less data to worry during >>> repair and that would help JAVA heap in general. You will have to do >>> multiple iterations to complete entire range but at-least it will work. >>> >>> -jaydeep >>> >>> >>> On Mon, Jun 30, 2014 at 3:22 PM, Robert Coli >>> wrote: >>> >>>> On Mon, Jun 30, 2014 at 3:08 PM, Yuki Morishita >>>> wrote: >>>> >>>>> Repair uses snapshot option by default since 2.0.2 (see NEWS.txt). >>>>> >>>> >>>> As a general meta comment, the process by which operationally important >>>> defaults change in Cassandra seems ad-hoc and sub-optimal. >>>> >>>> For to record, my view was that this change, which makes repair even >>>> slower than it previously was, was probably overly optimistic. >>>> >>>> It's also weird in that it changes default behavior which has been >>>> unchanged since the start of Cassandra time and is therefore probably >>>> automated against. Why was it so critically important to switch to snapshot >>>> repair that it needed to be shotgunned as a new default in 2.0.2? >>>> >>>> =Rob >>>> >>>> >>> >>> >> > -- *Paulo Motta* Chaordic | *Platform* *www.chaordic.com.br * +55 48 3232.3200 --001a11c1edec6d320a04fd17d4b7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
If you find it useful, I created a tool where you inp= ut the node IP, keyspace, column family, and optionally the number of parti= tions (default: 32K), and it outputs the list of subranges for that node, C= F, partition size:=C2=A0https://github.com/pauloricardomg/cassandra-list-subran= ges

So you can basically iterate over the output of that and do subran= ge repair for each node and cf, maybe in parallel. :)


On Mon, Jun 30, 2014 at= 10:26 PM, Phil Burress <philburresseme@gmail.com> wr= ote:
One last question. Any tips= on scripting a subrange repair?


On Mon, Jun 30, 2014 at 7:12 PM, Phil Bu= rress <philburresseme@gmail.com> wrote:
We are running repair -pr. = We've tried subrange manually and that seems to work ok. I guess we'= ;ll go with that going forward. Thanks for all the info!


On Mon, Jun 30, 2014 at 6:52 PM, Jaydeep Chovatia <chovatia.jayde= ep@gmail.com> wrote:
Are you running full repair or on subset? If you are runni= ng full repair then try running on sub-set of ranges which means less data = to worry during repair and that would help JAVA heap in general. You will h= ave to do multiple iterations to complete entire range but at-least it will= work.

-jaydeep


On Mon, Jun 30, 2014 at 3:22 = PM, Robert Coli <rcoli@eventbrite.com> wrote:
=
On Mon, Jun 30, 2014 at 3:08 PM, Yuki Moris= hita <mor.yuki@gmail.com> wrote:
Repair uses snapshot option by default since 2.= 0.2 (see NEWS.txt).

As a general meta comment, the = process by which operationally important defaults change in Cassandra seems= ad-hoc and sub-optimal.

For to record, my v= iew was that this change, which makes repair even slower than it previously= was, was probably overly optimistic.

It's also weird in that it changes default be= havior which has been unchanged since the start of Cassandra time and is th= erefore probably automated against. Why was it so critically important to s= witch to snapshot repair that it needed to be shotgunned as a new default i= n 2.0.2?

=3DRob
=C2=A0






--
=
Paulo = Motta

<= div style=3D"font-family:arial,sans-serif;font-size:12.727272033691406px;ba= ckground-color:rgb(255,255,255)">
Chaordic | Platform
www.chaordic.com.br
+55 48 3232.3200
--001a11c1edec6d320a04fd17d4b7--