Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AF9B618E20 for ; Thu, 18 Jun 2015 05:11:30 +0000 (UTC) Received: (qmail 8876 invoked by uid 500); 18 Jun 2015 05:11:27 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 8831 invoked by uid 500); 18 Jun 2015 05:11:27 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 8821 invoked by uid 99); 18 Jun 2015 05:11:27 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Jun 2015 05:11:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 302C9182238 for ; Thu, 18 Jun 2015 05:11:27 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.88 X-Spam-Level: ** X-Spam-Status: No, score=2.88 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id B97oM_PItVnP for ; Thu, 18 Jun 2015 05:11:18 +0000 (UTC) Received: from mail-wg0-f47.google.com (mail-wg0-f47.google.com [74.125.82.47]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id B2F6927650 for ; Thu, 18 Jun 2015 05:11:17 +0000 (UTC) Received: by wgfq1 with SMTP id q1so6542623wgf.1 for ; Wed, 17 Jun 2015 22:11:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ZtPw6RZVu1pof3W8Qfe7izAJRrAE9uNrCB7c/TIwOn0=; b=M1Q0xhugVh+3rYgrYZEWGtSanlFs74l2utEP/RrhGellLL/yQzM/cSAke9QdJYAe5+ y5yRb4dDkTdPIDEbpw69gnswQqDyPZHJn0vSYoMbfxZpXVWD7yojHSd5GMS+Y2Ik/fnC ks5VFN8iGzEvYN8EVFz3ZK08sBWzQWmHGPzmOPYJCcIIR8kYgAn2bSdf/xDEgSOXOUkQ 8mnYYYTy+GJIjJhuPjkW5S2mdP9GYkf02MB6qitTPswrJ5V6dh/MKj+jdB2QIRzuINot EmOoAsYM+fZFUTBY1v4jhK0NxCTWS5IpjYdS7OfhdhysD5xQ1nnhzWa6VJVGxBnwUVSt NjeA== MIME-Version: 1.0 X-Received: by 10.194.103.130 with SMTP id fw2mr11714853wjb.130.1434604270570; Wed, 17 Jun 2015 22:11:10 -0700 (PDT) Received: by 10.27.102.3 with HTTP; Wed, 17 Jun 2015 22:11:10 -0700 (PDT) In-Reply-To: References: Date: Thu, 18 Jun 2015 07:11:10 +0200 Message-ID: Subject: Re: Garbage collector launched on all nodes at once From: Marcus Eriksson To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=089e0102ef54f0d0910518c3d598 --089e0102ef54f0d0910518c3d598 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable It is probably this: https://issues.apache.org/jira/browse/CASSANDRA-9549 On Wed, Jun 17, 2015 at 7:37 PM, Micha=C5=82 =C5=81owicki wrote: > Looks that memtable heap size is growing on some nodes rapidly ( > https://www.dropbox.com/s/3brloiy3fqang1r/Screenshot%202015-06-17%2019.21= .49.png?dl=3D0). > Drops are the places when nodes have been restarted. > > On Wed, Jun 17, 2015 at 6:53 PM, Micha=C5=82 =C5=81owicki > wrote: > >> Hi, >> >> Two datacenters with 6 nodes (2.1.6) each. In each DC garbage collection >> is launched at the same time on each node (See [1] for total GC duration >> per 5 seconds). RF is set to 3. Any ideas? >> >> [1] >> https://www.dropbox.com/s/bsbyew1jxbe3dgo/Screenshot%202015-06-17%2018.4= 9.48.png?dl=3D0 >> >> -- >> BR, >> Micha=C5=82 =C5=81owicki >> > > > > -- > BR, > Micha=C5=82 =C5=81owicki > --089e0102ef54f0d0910518c3d598 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Jun 17, 2015 at 7:37 PM, Micha=C5=82 =C5=81owicki <mlowicki@gm= ail.com> wrote:
Looks that memtable heap size is growing on some nodes rapidly (https://www.dropbox.com/s/3brloiy3fqa= ng1r/Screenshot%202015-06-17%2019.21.49.png?dl=3D0). Drops are the plac= es when nodes have been restarted.

On Wed, Jun 17, 2015 at 6:53 = PM, Micha=C5=82 =C5=81owicki <mlowicki@gmail.com> wrote:
Hi,

Two= datacenters with 6 nodes (2.1.6) each. In each DC garbage collection is la= unched at the same time on each node (See [1] for total GC duration per 5 s= econds). RF is set to 3. Any ideas?




--
BR,
Micha=C5=82 =C5= =81owicki

--089e0102ef54f0d0910518c3d598--