Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 95061 invoked from network); 21 Jul 2010 13:17:13 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 21 Jul 2010 13:17:13 -0000 Received: (qmail 57816 invoked by uid 500); 21 Jul 2010 13:17:11 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 57669 invoked by uid 500); 21 Jul 2010 13:17:07 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 57661 invoked by uid 99); 21 Jul 2010 13:17:06 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Jul 2010 13:17:06 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates 209.85.214.172 as permitted sender) Received: from [209.85.214.172] (HELO mail-iw0-f172.google.com) (209.85.214.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Jul 2010 13:17:00 +0000 Received: by iwn38 with SMTP id 38so7522906iwn.31 for ; Wed, 21 Jul 2010 06:16:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type :content-transfer-encoding; bh=fsmN68EQt+jPbGPtfB/H75YJo53wLK5GWWOAsnR1g7A=; b=q+CiaOaa0zSUUTakXiHzawpFgrup0w2qTWK7FG0RaifcoT7AWfQ2VaTcgFxOmVPHrd 1DXRpuQKxbIh/G9fIkxIw4YTsO+ThWsKo+lyZJgpaQFcQ8b7ZxdCUhAuLqGxqx3M5SvF D6PP/C7kr0/kRCARTcsatSSOpUlbZyqmnJeOo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=AFlnTELBMBiSmnmO9nHX8Z+FYBcmOYyHQ1l2T4POQ4o4Qti7Hytmcj21YzfN81+JM6 kbuyaoxkEEuDpT/2N8GfXhGDIbg5WfQnCHEj5uFPOtOnEi/jINuXw0tFA6l7LvVO4UHW jHCG5WLZ/i1V8YMK3dFmQZFJB/dCLQuzLUt9Q= Received: by 10.231.33.67 with SMTP id g3mr179826ibd.31.1279718199092; Wed, 21 Jul 2010 06:16:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.194.77 with HTTP; Wed, 21 Jul 2010 06:16:18 -0700 (PDT) In-Reply-To: References: <3369e97d-ac19-e9d0-cc66-9a303b9e735c@me.com> <201007201447455523325@aspire-tech.com> <201007211229582822267@aspire-tech.com> From: Jonathan Ellis Date: Wed, 21 Jul 2010 06:16:18 -0700 Message-ID: Subject: Re: Re: What is consuming the heap? To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org On Tue, Jul 20, 2010 at 11:33 PM, Peter Schuller wrote: >> =A0INFO=A0[GC=A0inspection]=A02010-07-21=A001:01:49,661=A0GCInspector.ja= va=A0(line=A0110)=A0GC=A0for=A0ConcurrentMarkSweep:=A011748=A0ms,=A04136734= 72=A0reclaimed=A0leaving=A09779542600=A0used;=A0max=A0is=A010873667584 >> ERROR=A0[Thread-35]=A02010-07-21=A001:02:10,941=A0CassandraDaemon.java= =A0(line=A078)=A0Fatal=A0exception=A0in=A0thread=A0Thread[Thread-35,5,main] >> java.lang.OutOfMemoryError:=A0Java=A0heap=A0space >> =A0INFO=A0[GC=A0inspection]=A02010-07-21=A001:02:10,958=A0GCInspector.ja= va=A0(line=A0110)=A0GC=A0for=A0ConcurrentMarkSweep:=A010043=A0ms,=A0259576= =A0reclaimed=A0leaving=A010172790816=A0used;=A0max=A0is=A010873667584 > > So that confirms a "legitimate" out-of-memory condition in the sense > that CMS is reclaiming extremely little and the live set after a > concurrent mark/sweep is indeed around the 10 gig. Are you doing a lot of inserts? You might be hitting https://issues.apache.org/jira/browse/CASSANDRA-1042 --=20 Jonathan Ellis Project Chair, Apache Cassandra co-founder of Riptano, the source for professional Cassandra support http://riptano.com