Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 1862E200D08 for ; Thu, 21 Sep 2017 10:02:43 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 16C221609D0; Thu, 21 Sep 2017 08:02:43 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 5D93D1609B8 for ; Thu, 21 Sep 2017 10:02:42 +0200 (CEST) Received: (qmail 97322 invoked by uid 500); 21 Sep 2017 08:02:40 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 97294 invoked by uid 99); 21 Sep 2017 08:02:40 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Sep 2017 08:02:40 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id EF773D5CB3 for ; Thu, 21 Sep 2017 08:02:39 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.099 X-Spam-Level: X-Spam-Status: No, score=0.099 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id LtFu8eGNv5kP for ; Thu, 21 Sep 2017 08:02:38 +0000 (UTC) Received: from mail-io0-f171.google.com (mail-io0-f171.google.com [209.85.223.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 11FDB5FDD9 for ; Thu, 21 Sep 2017 08:02:38 +0000 (UTC) Received: by mail-io0-f171.google.com with SMTP id w94so9044652ioi.7 for ; Thu, 21 Sep 2017 01:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=e+/8h+5NEYYyjosr8x3PcPQB9Yv73FfF/RX5ntobQYk=; b=nfcIMqROPevI9F2Z+prqwKvY7YpsB+BPSwVkS87o4Dp+A4wEzsZtm7f3gR/VLlmKX1 l9v3YUFdsBXGpOBZwb2P/0RngnEQBGywwu/adEu8DxO7iV0bSUgOxMFBTEevKN09Stq9 gAS16ROHaxg3XmN3RD5oX39kGMUn4z6Ef3Z2kYMDEQWuwmdVtBTvblnXW/wsgmQUzkkf 1iehN09IP/5QtNjLZKBmAD9KOfkb1kOl+qlyDwJML1Lvo0C5QnVhFjtz6mvcgBxa33Di lF/alBKrKDRxhoEfmp3WsRgg+FrzAG4GUQZl39COu5OrC+CkmL+E4hFwDjzpo7yY1I5K PIxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=e+/8h+5NEYYyjosr8x3PcPQB9Yv73FfF/RX5ntobQYk=; b=YVY72GDbhY6JvNvNbpUCVScvnIbJnvwUgwQ18r2I/l0+9w1Bt2xbuymwhoJ8EuoVtE I8taFSIS7KowGyrWh4NhB1nrZDgGcaIkTyddAFNtVZsfbVHyVjPkXvcZ+vfUIjCT69zW dF/peQcqyUUTgEyewXU+gz6UFmPTMsW6s76rkBdpFH0JOuTq4Ny0+JqxOr1pouOw1OAQ yL9xmEDsn4SZBbsMZD7sF9giBLDV5kP7nkkvL/LDCaNRAzVRCpCWxYSWUuB9Y80Oalxo TiZc8YXJgqCBqb8i5MdeyIBbJV4B8jYEt5u4qY+V0EWDrxrWIrjD64qp7DPGYQNpnERq Qdwg== X-Gm-Message-State: AHPjjUixInTVg4a5ZUQrn6d7uOskivkxlTGlJz52XXzQKsZAMbuP1JCJ KqABuk3JZsBQt9HdbyHb2TCxn+R97w9IDXgqOuwjFg== X-Google-Smtp-Source: AOwi7QDnhqTeeWT1Uift2ARXQmwp5O6Bku/5VZ9Mba1RNcdw9olwMG03JACutpzIvfs5m9vG5GDPrdvkAImkE+jfRDk= X-Received: by 10.202.230.207 with SMTP id d198mr1438014oih.234.1505980956791; Thu, 21 Sep 2017 01:02:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.1.51 with HTTP; Thu, 21 Sep 2017 01:02:16 -0700 (PDT) In-Reply-To: References: From: Nicolas Guyomar Date: Thu, 21 Sep 2017 10:02:16 +0200 Message-ID: Subject: Re: Drastic increase in disk usage after starting repair on 3.7 To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary="001a1141ad8af7d76d0559ae8322" archived-at: Thu, 21 Sep 2017 08:02:43 -0000 --001a1141ad8af7d76d0559ae8322 Content-Type: text/plain; charset="UTF-8" Hi Paul, This might be a long shot, but some repairs might fail to clear their snapshot (not sure if its still the case with C* 3.7 however, I had the problem on 2.X branche). What does nodetool listsnapshot indicate ? On 21 September 2017 at 05:49, kurt greaves wrote: > repair does overstream by design, so if that node is inconsistent you'd > expect a bit of an increase. if you've got a backlog of compactions that's > probably due to repair and likely the cause of the increase. if you're > really worried you can rolling restart to stop the repair, otherwise maybe > try increasing compaction throughput. > --001a1141ad8af7d76d0559ae8322 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Paul,

This might be a long shot, but= some repairs might fail to clear their snapshot (not sure if its still the= case with C* 3.7 however, I had the problem on 2.X branche).
Wha= t does nodetool listsnapshot indicate ? =C2=A0

On 21 September 2017 at 05:49,= kurt greaves <kurt@instaclustr.com> wrote:
repair does overstream by design, so= if that node is inconsistent you'd expect a bit of an increase. if you= 've got a backlog of compactions that's probably due to repair and = likely the cause of the increase. if you're really worried you can roll= ing restart to stop the repair, otherwise maybe try increasing compaction t= hroughput.

--001a1141ad8af7d76d0559ae8322--