Return-Path: 
 <user-return-39481-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id BE3C010B62
	for <apmail-cassandra-user-archive@www.apache.org>;
 Sun,  9 Mar 2014 20:25:22 +0000 (UTC)
Received: (qmail 6504 invoked by uid 500); 9 Mar 2014 20:25:19 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 6417 invoked by uid 500); 9 Mar 2014 20:25:19 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 6409 invoked by uid 99); 9 Mar 2014 20:25:19 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Mar 2014 20:25:19 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of tsato@cloudian.com designates
 209.85.213.177 as permitted sender)
Received: from [209.85.213.177] (HELO mail-ig0-f177.google.com)
 (209.85.213.177)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 09 Mar 2014 20:25:15 +0000
Received: by mail-ig0-f177.google.com with SMTP id ur14so6589368igb.4
        for <user@cassandra.apache.org>; Sun, 09 Mar 2014 13:24:54 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:content-type;
        bh=Vh3++OrFHQdnHNRpCM8KKQTGLH4fSTawhb/vFpEpowE=;
        b=h2G5Lr/F0etSrO4YTAt50wR2VoDwUl28VOBlvcCdKI5DrEaeHAevM8G+DpEy2epIK5
         GKqSHzFqkyjxx8XyZT4nUN9hQ2fDApPhsQNgktpqpvZPZJtkm2ZbhzMt28hvB7IMy3Py
         +rOSVQJjJzKBSsLk2/m5GbSiyihj7x3s2ptCaor8/Kae9T5V46UQFzaHj7ZsUSLQRFi9
         SRhVDldCZ1QOnHMy/6Sbx0M77zhEBz7349U3oQgf3NU6Ibb6K5Da3JRjKQtCs5JOy5BF
         8Km+QHzz96pybO/Z8ZoHbxqPYQUVOxOlTAKqAVgKnk+mkphrYKr9RPA4WeBflppXMCrb
         1tUA==
X-Gm-Message-State: 
 ALoCoQk1kKyDnyAFVGXitncM86EKfA7g07N0uxI2o+ko2v8U3L2b/frqqK83LVQXY2cYpB4hZ0j3
MIME-Version: 1.0
X-Received: by 10.42.16.199 with SMTP id q7mr25089878ica.16.1394396694202;
 Sun, 09 Mar 2014 13:24:54 -0700 (PDT)
Received: by 10.64.73.41 with HTTP; Sun, 9 Mar 2014 13:24:54 -0700 (PDT)
In-Reply-To: <lfhrst$hhc$1@ger.gmane.org>
References: <lfhrst$hhc$1@ger.gmane.org>
Date: Mon, 10 Mar 2014 05:24:54 +0900
Message-ID: 
 <CAFWg+hCXUeFseEuunRvd56ExxdJL0dzntjjz8+aycZ8km6gP3g@mail.gmail.com>
Subject: Re: need help with Cassandra 1.2 Full GCing -- output of jmap
 histogram
From: Takenori Sato <tsato@cloudian.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=20cf303f6674a266ba04f43247dd
X-Virus-Checked: Checked by ClamAV on apache.org

--20cf303f6674a266ba04f43247dd
Content-Type: text/plain; charset=ISO-8859-1

You have millions of org.apache.cassandra.db.DeletedColumn instances on the
snapshot.

This means you have lots of column tombstones, and I guess, which are read
into memory by slice query.


On Sun, Mar 9, 2014 at 10:55 PM, Oleg Dulin <oleg.dulin@gmail.com> wrote:

> I am trying to understand why one of my nodes keeps full GC.
>
> I have Xmx set to 8gigs, memtable total size is 2 gigs.
>
> Consider the top entries from jmap -histo:live @
> http://pastebin.com/UaatHfpJ
>
> --
> Regards,
> Oleg Dulin
> http://www.olegdulin.com
>
>
>

--20cf303f6674a266ba04f43247dd
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">You have millions of=A0<span style=3D"color:rgb(0,0,0);fon=
t-family:Consolas,Menlo,Monaco,&#39;Lucida Console&#39;,&#39;Liberation Mon=
o&#39;,&#39;DejaVu Sans Mono&#39;,&#39;Bitstream Vera Sans Mono&#39;,monosp=
ace,serif;font-size:12px;line-height:21px">org.apache.cassandra.db.DeletedC=
olumn</span>=A0instances on the snapshot.<div>
<br></div><div>This means you have lots of column tombstones, and I guess, =
which are read into memory by slice query.=A0</div></div><div class=3D"gmai=
l_extra"><br><br><div class=3D"gmail_quote">On Sun, Mar 9, 2014 at 10:55 PM=
, Oleg Dulin <span dir=3D"ltr">&lt;<a href=3D"mailto:oleg.dulin@gmail.com" =
target=3D"_blank">oleg.dulin@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">I am trying to understand why one of my node=
s keeps full GC.<br>
<br>
I have Xmx set to 8gigs, memtable total size is 2 gigs.<br>
<br>
Consider the top entries from jmap -histo:live @ <a href=3D"http://pastebin=
.com/UaatHfpJ" target=3D"_blank">http://pastebin.com/UaatHfpJ</a><span clas=
s=3D"HOEnZb"><font color=3D"#888888"><br>
<br>
-- <br>
Regards,<br>
Oleg Dulin<br>
<a href=3D"http://www.olegdulin.com" target=3D"_blank">http://www.olegdulin=
.com</a><br>
<br>
<br>
</font></span></blockquote></div><br></div>

--20cf303f6674a266ba04f43247dd--