Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of rcoli@eventbrite.com designates
 209.85.220.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAF+j8akbbvFWHd0pxmaXmH5kXihgdPLcUn1T1ojUxRAzzwHFJw@mail.gmail.com>
References: 
 <CAF+j8akbbvFWHd0pxmaXmH5kXihgdPLcUn1T1ojUxRAzzwHFJw@mail.gmail.com>
Date: Wed, 5 Nov 2014 12:10:34 -0800
Message-ID: 
 <CAEDUwd2aUPBkbEMXBQQ_QUN+n=JEh8WuOtNqO3N0gr9ejBWDSg@mail.gmail.com>
Subject: Re: Cassandra heap pre-1.1
From: Robert Coli <rcoli@eventbrite.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=089e013cb9be263c0e0507222cdd

--089e013cb9be263c0e0507222cdd
Content-Type: text/plain; charset=UTF-8

On Tue, Nov 4, 2014 at 8:51 PM, Raj N <raj.cassandra@gmail.com> wrote:

> Is there a good formula to calculate heap utilization in Cassandra
> pre-1.1, specifically 1.0.10. We are seeing gc pressure on our nodes. And I
> am trying to estimate what could be causing this? Using node tool info my
> steady state heap is at about 10GB. XMX is 12G.
>

Basically, no. If you really want to know, take a heap dump and load it
into Eclipse Memory Analyzer.


> I have 4.5 GB of bloom filters which I can derive looking at cfstats
>

This is a *very* large percentage of your total heap, and is probably the
lever you have most influence on pulling.


> I have negligible row caching.
>

Row caching is generally not advised in that era, especially with heap
pressure.


> I have key caching enabled on my cfs. I couldn't find an easy way to
> estimate how much this is using, but I tried to invalidate the key cache
> and I got 1.3 GB back.
>

Key caching is generally advisable, but 1.3GB is a lot of key cache..


> That still only adds up to 5.8 GB. I know there is index sampling going on
> as well. I have around 800 million rows. Is there a way to estimate how
> much space this would add up to?
>

Plenty. You should reduce your bloom filter size, or upgrade to a version
of Cassandra that moves stuff off the heap.

=Rob
http://twitter.com/rcolidba

--089e013cb9be263c0e0507222cdd
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On T=
ue, Nov 4, 2014 at 8:51 PM, Raj N <span dir=3D"ltr">&lt;<a href=3D"mailto:r=
aj.cassandra@gmail.com" target=3D"_blank">raj.cassandra@gmail.com</a>&gt;</=
span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Is there a =
good formula to calculate heap utilization in Cassandra pre-1.1, specifical=
ly 1.0.10. We are seeing gc pressure on our nodes. And I am trying to estim=
ate what could be causing this? Using node tool info my steady state heap i=
s at about 10GB. XMX is 12G.</div></blockquote><div><br></div><div>Basicall=
y, no. If you really want to know, take a heap dump and load it into Eclips=
e Memory Analyzer.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" =
style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><di=
v dir=3D"ltr"><div>I have 4.5 GB of bloom filters which I can derive lookin=
g at cfstats</div></div></blockquote><div><br></div><div>This is a *very* l=
arge percentage of your total heap, and is probably the lever you have most=
 influence on pulling.</div><div>=C2=A0</div><blockquote class=3D"gmail_quo=
te" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"=
><div dir=3D"ltr"><div>I have negligible row caching.</div></div></blockquo=
te><div><br></div><div>Row caching is generally not advised in that era, es=
pecially with heap pressure.</div><div>=C2=A0</div><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex"><div dir=3D"ltr"><div>I have key caching enabled on my cfs. I couldn=
&#39;t find an easy way to estimate how much this is using, but I tried to =
invalidate the key cache and I got 1.3 GB back.</div></div></blockquote><di=
v><br></div><div>Key caching is generally advisable, but 1.3GB is a lot of =
key cache..</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=
=3D"ltr"><div>That still only adds up to 5.8 GB. I know there is index samp=
ling going on as well. I have around 800 million rows. Is there a way to es=
timate how much space this would add up to?</div></div></blockquote><div><b=
r></div><div>Plenty. You should reduce your bloom filter size, or upgrade t=
o a version of Cassandra that moves stuff off the heap.</div><div><br></div=
><div>=3DRob</div><div><a href=3D"http://twitter.com/rcolidba">http://twitt=
er.com/rcolidba</a></div><div><br></div><div>=C2=A0</div></div></div></div>

--089e013cb9be263c0e0507222cdd--