Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from
	:mime-version:content-type:subject:date:in-reply-to:to
	:references:message-id; q=dns; s=thelastpickle.com; b=w8rQIuquFX
	AFbBukcKR/17ZKcnPdizJ1rMg1y2+ugUnHNjoBxXkWbmhexta7g+dA36qXkgPxPI
	up/otraWZoHfoCTa2iukI6FwuFNyM2MYSYgPlVvBTNKc2Fd5kTjY/oMlIEKAdYXX
	W23x7pW4J/5v7XV+ZGGlZ91lc13N+mQYQ=
From: aaron morton <aaron@thelastpickle.com>
Mime-Version: 1.0 (Apple Message framework v1081)
Content-Type: multipart/alternative; boundary=Apple-Mail-1-605024484
Subject: Re: Flush and compaction happens frequently
Date: Tue, 7 Sep 2010 22:21:42 +1200
In-Reply-To: <AANLkTi=YRvGoexm8np_11StXQTEeozLk2TRJtDq=A9zm@mail.gmail.com>
To: user@cassandra.apache.org
References: <AANLkTi=YRvGoexm8np_11StXQTEeozLk2TRJtDq=A9zm@mail.gmail.com>
Message-Id: <B480025B-C263-489F-8159-F1687EAE9E15@thelastpickle.com>


--Apple-Mail-1-605024484
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

That sounds a little odd, may help if you post the yaml settings, the=20
tpstats and the log lines that look something like this...

INFO [ROW-MUTATION-STAGE:5] 2010-09-04 15:43:49,402=20
ColumnFamilyStore.java (line 790) Enqueuing flush of=20
Memtable-Super1@1754565178(80208 bytes, 2304 operations)

Also wondering why you chose such high memtable thresholds and how=20
that's working. Do you expect to trigger the operation count or the
throughput count in normal processing? Have you seen the additional=20
guidance on mem table tuning here=20
http://wiki.apache.org/cassandra/MemtableThresholds

Aaron

On 7 Sep 2010, at 18:33, Mubarak Seyed wrote:

> I have a 8 nodes cluster, MemtableThreshold is 2 GB/CF, =
MemtableObjectsCount is 1.2, heap (min/max) is 30 GB and only 4 =
ColumnFamilies
>=20
> It appears from system.log that flush happens for every < 50 =
operations (read or write) and compaction is happening very frequently =
and i could see lots of sstable is getting created (with smaller size). =
For a just 1000 inserts, i could see around 20 sstables.
>=20
> When i change the MemtableThreshold to 1 GB/CF, everything works as =
desired.
>=20
> Any idea, what could be the problem when i specify MemtableThreshold =
to 2 GB/CF even though i specified the large heap?
>=20
> --=20
> Thanks,
> Mubarak Seyed.


--Apple-Mail-1-605024484
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">That =
sounds a little odd, may help if you post the yaml settings, =
the&nbsp;<br>tpstats and the log lines that look something like =
this...<br><br>INFO [ROW-MUTATION-STAGE:5] 2010-09-04 =
15:43:49,402&nbsp;<br>ColumnFamilyStore.java (line 790) Enqueuing flush =
of&nbsp;<br>Memtable-Super1@1754565178(80208 bytes, 2304 =
operations)<br><br>Also wondering why you chose such high memtable =
thresholds and how&nbsp;<br>that's working. Do you expect to trigger the =
operation count or the<br>throughput count in normal processing? Have =
you seen the additional&nbsp;<br>guidance on mem table tuning =
here&nbsp;<br><a =
href=3D"http://wiki.apache.org/cassandra/MemtableThresholds">http://wiki.a=
pache.org/cassandra/MemtableThresholds</a><br><br>Aaron<div><br><div><div>=
On 7 Sep 2010, at 18:33, Mubarak Seyed wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite">I have a 8 =
nodes cluster, MemtableThreshold is 2 GB/CF, MemtableObjectsCount is =
1.2, heap (min/max) is 30 GB and only 4 =
ColumnFamilies<div><br></div><div>It appears from system.log that flush =
happens for every &lt; 50 operations (read or write) and compaction is =
happening very frequently and i could see lots of sstable is getting =
created (with smaller size). For a just 1000 inserts, i could see around =
20 sstables.</div>
<div><br></div><div>When i change the MemtableThreshold to 1 GB/CF, =
everything works as desired.</div><div><br></div><div>Any idea, what =
could be the problem when i specify MemtableThreshold to 2 GB/CF even =
though i specified the large heap?<br clear=3D"all">
<br>-- <br>Thanks,<br>Mubarak Seyed.<br>
</div>
</blockquote></div><br></div></body></html>=

--Apple-Mail-1-605024484--