Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 18065 invoked from network); 2 Jun 2010 16:36:30 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Jun 2010 16:36:30 -0000 Received: (qmail 56037 invoked by uid 500); 2 Jun 2010 16:36:29 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 56020 invoked by uid 500); 2 Jun 2010 16:36:29 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 56012 invoked by uid 99); 2 Jun 2010 16:36:29 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jun 2010 16:36:29 +0000 X-ASF-Spam-Status: No, hits=1.3 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ryan@twitter.com designates 74.125.83.44 as permitted sender) Received: from [74.125.83.44] (HELO mail-gw0-f44.google.com) (74.125.83.44) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jun 2010 16:36:23 +0000 Received: by gwj19 with SMTP id 19so4896653gwj.31 for ; Wed, 02 Jun 2010 09:36:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.224.19.88 with SMTP id z24mr1742713qaa.399.1275496556444; Wed, 02 Jun 2010 09:35:56 -0700 (PDT) Received: by 10.229.3.147 with HTTP; Wed, 2 Jun 2010 09:35:56 -0700 (PDT) In-Reply-To: <1275434440959-5128481.post@n2.nabble.com> References: <1275434440959-5128481.post@n2.nabble.com> Date: Wed, 2 Jun 2010 09:35:56 -0700 Message-ID: Subject: Re: Nodes dropping out of cluster due to GC From: Ryan King To: user@cassandra.apache.org Cc: cassandra-user@incubator.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Why run with so few nodes? -ryan On Tue, Jun 1, 2010 at 4:20 PM, Eric Halpern wrote: > > Hello, > > We're running a 4 node cluster on beefy EC2 virtual instances (8 core, 32 > GB) using EBS storage with 8 GB of heap allocated to the JVM. > > Every couple of hours, each of the nodes does a concurrent mark/sweep tha= t > takes around 30 seconds to complete. =A0During that GC, the node temporar= ily > drops out of the cluster, usually for about 15 seconds. > > The frequency of the concurrent mark sweeps seems reasonable, but the fac= t > that the node drops out of the cluster temporarily is a major problem sin= ce > this has significant impact on the performance and stability of our servi= ce. > > Has anyone experienced this sort of problem? =A0It would be great to hear= from > anyone who has had experience with this sort of issue and/or suggestions = for > how to deal with it. > > Thanks, Eric > -- > View this message in context: http://cassandra-user-incubator-apache-org.= 3065146.n2.nabble.com/Nodes-dropping-out-of-cluster-due-to-GC-tp5128481p512= 8481.html > Sent from the cassandra-user@incubator.apache.org mailing list archive at= Nabble.com. >