Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 41629 invoked from network); 2 Jun 2010 19:46:03 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Jun 2010 19:46:03 -0000 Received: (qmail 70871 invoked by uid 500); 2 Jun 2010 19:46:02 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 70849 invoked by uid 500); 2 Jun 2010 19:46:02 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 70841 invoked by uid 500); 2 Jun 2010 19:46:02 -0000 Delivered-To: apmail-incubator-cassandra-user@incubator.apache.org Received: (qmail 70837 invoked by uid 99); 2 Jun 2010 19:46:02 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jun 2010 19:46:01 +0000 X-ASF-Spam-Status: No, hits=2.3 required=10.0 tests=SPF_HELO_PASS,SPF_SOFTFAIL,URI_HEX X-Spam-Check-By: apache.org Received-SPF: softfail (nike.apache.org: transitioning domain of eric@dnagamesinc.com does not designate 216.139.236.158 as permitted sender) Received: from [216.139.236.158] (HELO kuber.nabble.com) (216.139.236.158) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jun 2010 19:45:55 +0000 Received: from jim.nabble.com ([192.168.236.80]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1OJtsd-0005Eq-1I for cassandra-user@incubator.apache.org; Wed, 02 Jun 2010 12:45:35 -0700 Date: Wed, 2 Jun 2010 12:45:35 -0700 (PDT) From: Eric Halpern To: cassandra-user@incubator.apache.org Message-ID: <1275507935034-5132279.post@n2.nabble.com> In-Reply-To: References: <1275434440959-5128481.post@n2.nabble.com> Subject: Re: Nodes dropping out of cluster due to GC MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Ryan King wrote: >=20 > Why run with so few nodes? >=20 > -ryan >=20 > On Tue, Jun 1, 2010 at 4:20 PM, Eric Halpern wrote= : >> >> Hello, >> >> We're running a 4 node cluster on beefy EC2 virtual instances (8 core, 3= 2 >> GB) using EBS storage with 8 GB of heap allocated to the JVM. >> >> Every couple of hours, each of the nodes does a concurrent mark/sweep >> that >> takes around 30 seconds to complete. =C2=A0During that GC, the node >> temporarily >> drops out of the cluster, usually for about 15 seconds. >> >> The frequency of the concurrent mark sweeps seems reasonable, but the >> fact >> that the node drops out of the cluster temporarily is a major problem >> since >> this has significant impact on the performance and stability of our >> service. >> >> Has anyone experienced this sort of problem? =C2=A0It would be great to = hear >> from >> anyone who has had experience with this sort of issue and/or suggestions >> for >> how to deal with it. >> >> Thanks, Eric >> -- >=20 >=20 We wanted to start with a small number of nodes to test things out before going big. Is there some reason that a small cluster would cause more problems in this regard. The actual request load is actually pretty light for the cluster. --=20 View this message in context: http://cassandra-user-incubator-apache-org.30= 65146.n2.nabble.com/Nodes-dropping-out-of-cluster-due-to-GC-tp5128481p51322= 79.html Sent from the cassandra-user@incubator.apache.org mailing list archive at N= abble.com.