Return-Path: 
 <user-return-10753-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: (qmail 76107 invoked from network); 26 Nov 2010 14:17:52 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 26 Nov 2010 14:17:52 -0000
Received: (qmail 16824 invoked by uid 500); 26 Nov 2010 14:17:50 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 16553 invoked by uid 500); 26 Nov 2010 14:17:50 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 16538 invoked by uid 99); 26 Nov 2010 14:17:49 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Nov 2010 14:17:49 +0000
X-ASF-Spam-Status: No, hits=2.2 required=10.0
	tests=HTML_MESSAGE,SPF_PASS,T_HK_NAME_DR
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of ajp@eutechnyx.com designates
 194.116.198.142 as permitted sender)
Received: from [194.116.198.142] (HELO gse-mta-03.emailfiltering.com)
 (194.116.198.142)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Nov 2010 14:17:44 +0000
Received: from host-195-188-251-108.reversezone.co.uk ([195.188.251.108])
	by gse-mta-03.emailfiltering.com with emfmta (version 4.6.0.72) vanilla id
 9592857722
	for user@cassandra.apache.org;8149352552e4c4f5;
 Fri, 26 Nov 2010 14:17:21 +0000
Received: from xchange.eutechnyx.com (10.12.0.40:14868)
	by mail.eutechnyx.com with [XMail 1.20 ESMTP Server]
	id <S4B89C9> for <user@cassandra.apache.org> from <ajp@eutechnyx.com>;
	Fri, 26 Nov 2010 14:17:20 -0000
Received: from XCHANGE.eutechnyx.com ([::1]) by xchange.eutechnyx.com ([::1])
 with mapi; Fri, 26 Nov 2010 14:17:19 +0000
From: "Dr. Andrew Perella" <ajp@eutechnyx.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Subject: RE: Newbie question on Cassandra mem usage
Thread-Topic: Newbie question on Cassandra mem usage
Thread-Index: AQHLinWBXJ2oI3qvP0ybvkwWOcmRvZN9624AgAAWRoCAAATgAIAFzaAw
Date: Fri, 26 Nov 2010 14:17:16 +0000
Message-ID: <9F83B51B5E3935419B14FC2D1436FC094FE3F55A@xchange.eutechnyx.com>
References: <AANLkTi=KsxNEg892T3Nc956hCyMw_rSgsEb-_gFtQ+pC@mail.gmail.com>
 <e6727c3d-6b0d-60d1-b6ea-d0712d66ae21@me.com>
In-Reply-To: <e6727c3d-6b0d-60d1-b6ea-d0712d66ae21@me.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Content-Type: multipart/alternative;
	boundary="_000_9F83B51B5E3935419B14FC2D1436FC094FE3F55Axchangeeutechny_"
MIME-Version: 1.0

--_000_9F83B51B5E3935419B14FC2D1436FC094FE3F55Axchangeeutechny_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

How can I set these per CF when I create them dynamically?
Regards,
Andrew

From: Aaron Morton [mailto:aaron@thelastpickle.com]
Sent: 22 November 2010 21:40
To: user@cassandra.apache.org
Subject: Re: Newbie question on Cassandra mem usage

They are memtable_throughput_in_mb, memtable_flush_after_mins, memtable_ope=
rations_in_millions.

Under 0.7 these are per CF settings, in 0.6 these are cluster wide.

To start with try  mb one down to something like 64 or 128, ops to 0.5 and =
mins to 60 .

What version are you using ?

Aaron


On 23 Nov, 2010,at 10:22 AM, Trung Tran <tran.hieutrung@gmail.com> wrote:
Hi,

Is it the min_compaction_threshold and max_compaction_threshold? Do i
need to lower the memtable setting also?

Thanks,
Trung.

On Mon, Nov 22, 2010 at 12:02 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
> Set your columnfamily thresholds lower.
>
> On Mon, Nov 22, 2010 at 12:45 PM, Trung Tran <tran.hieutrung@gmail.com> w=
rote:
>> Hi,
>>
>> I have a test cluster of 3 nodes, 14Gb of mem in each node,
>> replication factor =3D 3. With default -Xms and Xmx, my nodes are set to
>> have max-heap-size =3D 7Gb. After initial load with about 200M rows
>> (write with hector default consistencylevel =3D quorum,) my nodes memory
>> usage are up to 13.5Gb, show a bunches of GC notifications and
>> eventually crashes with java.lang.OutOfMemoryError: Java heap space.
>>
>> Is there any setting that can help with this scenario?
>>
>> Thanks,
>> Trung.
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>

This email has been scanned for all known viruses by the Email Protection A=
gency
=
This e-mail is confidential and may be privileged. It may be read, copi=
ed and used only by the intended recipient. No communication sent by e-=
mail to or from Eutechnyx is intended to give rise to contractual or ot=
her legal liability, apart from liability which cannot be excluded unde=
r English law.=20=0D=0A=0D=0AThis email has been scanned for all known =
viruses by the Email Protection Agency. http://www.epagency.net=0D=0A=0D=
=0A=0D=0Awww.eutechnyx.com Eutechnyx Limited. Registered in England No:=
 2172322=0D=0A=

--_000_9F83B51B5E3935419B14FC2D1436FC094FE3F55Axchangeeutechny_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:x=3D"urn:schemas-microsoft-com:office:excel" xmlns:p=3D"urn:schemas-m=
icrosoft-com:office:powerpoint" xmlns:a=3D"urn:schemas-microsoft-com:office=
:access" xmlns:dt=3D"uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s=3D"=
uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs=3D"urn:schemas-microsof=
t-com:rowset" xmlns:z=3D"#RowsetSchema" xmlns:b=3D"urn:schemas-microsoft-co=
m:office:publisher" xmlns:ss=3D"urn:schemas-microsoft-com:office:spreadshee=
t" xmlns:c=3D"urn:schemas-microsoft-com:office:component:spreadsheet" xmlns=
:odc=3D"urn:schemas-microsoft-com:office:odc" xmlns:oa=3D"urn:schemas-micro=
soft-com:office:activation" xmlns:html=3D"http://www.w3.org/TR/REC-html40" =
xmlns:q=3D"http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc=3D"http://m=
icrosoft.com/officenet/conferencing" xmlns:D=3D"DAV:" xmlns:Repl=3D"http://=
schemas.microsoft.com/repl/" xmlns:mt=3D"http://schemas.microsoft.com/share=
point/soap/meetings/" xmlns:x2=3D"http://schemas.microsoft.com/office/excel=
/2003/xml" xmlns:ppda=3D"http://www.passport.com/NameSpace.xsd" xmlns:ois=
=3D"http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir=3D"http://=
schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds=3D"http://www.w3=
.org/2000/09/xmldsig#" xmlns:dsp=3D"http://schemas.microsoft.com/sharepoint=
/dsp" xmlns:udc=3D"http://schemas.microsoft.com/data/udc" xmlns:xsd=3D"http=
://www.w3.org/2001/XMLSchema" xmlns:sub=3D"http://schemas.microsoft.com/sha=
repoint/soap/2002/1/alerts/" xmlns:ec=3D"http://www.w3.org/2001/04/xmlenc#"=
 xmlns:sp=3D"http://schemas.microsoft.com/sharepoint/" xmlns:sps=3D"http://=
schemas.microsoft.com/sharepoint/soap/" xmlns:xsi=3D"http://www.w3.org/2001=
/XMLSchema-instance" xmlns:udcs=3D"http://schemas.microsoft.com/data/udc/so=
ap" xmlns:udcxf=3D"http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udc=
p2p=3D"http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf=3D"http:/=
/schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss=3D"http://sche=
mas.microsoft.com/office/2006/digsig-setup" xmlns:dssi=3D"http://schemas.mi=
crosoft.com/office/2006/digsig" xmlns:mdssi=3D"http://schemas.openxmlformat=
s.org/package/2006/digital-signature" xmlns:mver=3D"http://schemas.openxmlf=
ormats.org/markup-compatibility/2006" xmlns:m=3D"http://schemas.microsoft.c=
om/office/2004/12/omml" xmlns:mrels=3D"http://schemas.openxmlformats.org/pa=
ckage/2006/relationships" xmlns:spwp=3D"http://microsoft.com/sharepoint/web=
partpages" xmlns:ex12t=3D"http://schemas.microsoft.com/exchange/services/20=
06/types" xmlns:ex12m=3D"http://schemas.microsoft.com/exchange/services/200=
6/messages" xmlns:pptsl=3D"http://schemas.microsoft.com/sharepoint/soap/Sli=
deLibrary/" xmlns:spsl=3D"http://microsoft.com/webservices/SharePointPortal=
Server/PublishedLinksService" xmlns:Z=3D"urn:schemas-microsoft-com:" xmlns:=
st=3D"&#1;" xmlns=3D"http://www.w3.org/TR/REC-html40">

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)">
<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
p
	{mso-style-priority:99;
	mso-margin-top-alt:auto;
	margin-right:0cm;
	mso-margin-bottom-alt:auto;
	margin-left:0cm;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
span.EmailStyle18
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext=3D"edit">
  <o:idmap v:ext=3D"edit" data=3D"1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body lang=3DEN-GB link=3Dblue vlink=3Dpurple>

<div class=3DWordSection1>

<p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri",=
"sans-serif";
color:#1F497D'>How can I set these per CF when I create them dynamically?<o=
:p></o:p></span></p>

<p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri",=
"sans-serif";
color:#1F497D'>Regards,<o:p></o:p></span></p>

<p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri",=
"sans-serif";
color:#1F497D'>Andrew<o:p></o:p></span></p>

<p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri",=
"sans-serif";
color:#1F497D'><o:p>&nbsp;</o:p></span></p>

<div>

<div style=3D'border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm =
0cm 0cm'>

<p class=3DMsoNormal><b><span lang=3DEN-US style=3D'font-size:10.0pt;font-f=
amily:
"Tahoma","sans-serif"'>From:</span></b><span lang=3DEN-US style=3D'font-siz=
e:10.0pt;
font-family:"Tahoma","sans-serif"'> Aaron Morton
[mailto:aaron@thelastpickle.com] <br>
<b>Sent:</b> 22 November 2010 21:40<br>
<b>To:</b> user@cassandra.apache.org<br>
<b>Subject:</b> Re: Newbie question on Cassandra mem usage<o:p></o:p></span=
></p>

</div>

</div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

<div>

<p class=3DMsoNormal>They are memtable_throughput_in_mb,
memtable_flush_after_mins, memtable_operations_in_millions.&nbsp;<o:p></o:p=
></p>

</div>

<div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

</div>

<div>

<p class=3DMsoNormal>Under 0.7 these are per CF settings, in 0.6 these are
cluster wide.&nbsp;<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

</div>

<div>

<p class=3DMsoNormal>To start with try &nbsp;mb one down to something like =
64 or
128, ops to 0.5 and mins to 60 .&nbsp;<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

</div>

<div>

<p class=3DMsoNormal>What version are you using ?&nbsp;<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal><o:p>&nbsp;</o:p></p>

</div>

<div>

<p class=3DMsoNormal>Aaron<o:p></o:p></p>

</div>

<div>

<p class=3DMsoNormal style=3D'margin-bottom:12.0pt'><br>
<br>
On 23 Nov, 2010,at 10:22 AM, Trung Tran &lt;tran.hieutrung@gmail.com&gt; wr=
ote:<o:p></o:p></p>

</div>

<div>

<blockquote style=3D'margin-top:5.0pt;margin-bottom:5.0pt'>

<div>

<div>

<p class=3DMsoNormal>Hi,<br>
<br>
Is it the min_compaction_threshold and max_compaction_threshold? Do i<br>
need to lower the memtable setting also?<br>
<br>
Thanks,<br>
Trung.<br>
<br>
On Mon, Nov 22, 2010 at 12:02 PM, Jonathan Ellis &lt;jbellis@gmail.com&gt;
wrote:<br>
&gt; Set your columnfamily thresholds lower.<br>
&gt;<br>
&gt; On Mon, Nov 22, 2010 at 12:45 PM, Trung Tran
&lt;tran.hieutrung@gmail.com&gt; wrote:<br>
&gt;&gt; Hi,<br>
&gt;&gt;<br>
&gt;&gt; I have a test cluster of 3 nodes, 14Gb of mem in each node,<br>
&gt;&gt; replication factor =3D 3. With default -Xms and Xmx, my nodes are =
set to<br>
&gt;&gt; have max-heap-size =3D 7Gb. After initial load with about 200M row=
s<br>
&gt;&gt; (write with hector default consistencylevel =3D quorum,) my nodes =
memory<br>
&gt;&gt; usage are up to 13.5Gb, show a bunches of GC notifications and<br>
&gt;&gt; eventually crashes with java.lang.OutOfMemoryError: Java heap spac=
e.<br>
&gt;&gt;<br>
&gt;&gt; Is there any setting that can help with this scenario?<br>
&gt;&gt;<br>
&gt;&gt; Thanks,<br>
&gt;&gt; Trung.<br>
&gt;&gt;<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; --<br>
&gt; Jonathan Ellis<br>
&gt; Project Chair, Apache Cassandra<br>
&gt; co-founder of Riptano, the source for professional Cassandra support<b=
r>
&gt; <a href=3D"http://riptano.com">http://riptano.com</a><br>
&gt;<o:p></o:p></p>

</div>

</div>

</blockquote>

</div>

<p><span style=3D'font-size:7.5pt;font-family:"Arial","sans-serif"'>This em=
ail
has been scanned for all known viruses by the Email Protection Agency<o:p><=
/o:p></span></p>

</div>

=
<p><font face=3D"Arial" size=3D1>This e-mail is confidential and may be=
=20=0D=0Aprivileged. It may be read, copied and used only by the intend=
ed recipient. No=20=0D=0Acommunication sent by e-mail to or from Eutech=
nyx is intended to give rise to=20=0D=0Acontractual or other legal liab=
ility, apart from liability which cannot be=20=0D=0Aexcluded under Engl=
ish law. </font></p>=0D=0A<p><font face=3D"Arial" size=3D1>This email h=
as been scanned for all known viruses by <a href=3D"http://www.epagency=
.net">www.epagency.net</a>.</p>=20=0D=0A<a href=3D"http://www.eutechnyx=
.com">www.eutechnyx.com</a>=0D=0A</font><font face=3D"Arial" size=3D1>E=
utechnyx Limited. Registered in England No:=20=0D=0A2172322</font></p>=0D=
=0A=
</body>

</html>

--_000_9F83B51B5E3935419B14FC2D1436FC094FE3F55Axchangeeutechny_--