Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=dSmujHRuIb3MqeIkuxhWN8ZkKE9sc0WxvGc2Tm7f6UONb8vh0NoRPvBo+2iJLYRsEAD1Gd5q2vLZTY6p8Nkdc0AA0rcyy6lStjzDDrADQZCaSKRxK9Rt100/wqhm+bmdiOnl2DwS2HwFGCXUMfhDB3aOFRhNc+DlMKaX2CqcsuU=;
Message-ID: <614727.28922.qm@web55204.mail.re4.yahoo.com>
References: <249809.22549.qm@web55201.mail.re4.yahoo.com>
 <AANLkTikP4r-ydhWBKk2hzQ6QVCE1ZYx_DFt7RNveTugS@mail.gmail.com>
Date: Sun, 12 Sep 2010 16:10:01 -0700 (PDT)
From: kannan chandrasekaran <ckannanck@yahoo.com>
Subject: Re: Couple of cache related questions
To: user@cassandra.apache.org
In-Reply-To: <AANLkTikP4r-ydhWBKk2hzQ6QVCE1ZYx_DFt7RNveTugS@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-1819829274-1284333001=:28922"

--0-1819829274-1284333001=:28922
Content-Type: text/plain; charset=us-ascii

Thanks for the replies Jonathan...Couple more clarifications(in bold)


________________________________
From: Jonathan Ellis <jbellis@gmail.com>
To: user@cassandra.apache.org
Sent: Sun, September 12, 2010 1:47:09 PM
Subject: Re: Couple of cache related questions

On Sun, Sep 12, 2010 at 2:56 PM, kannan chandrasekaran
<ckannanck@yahoo.com> wrote:
> 1) What determines the amount of memory used per schema ignoring the general
> overhead to get cassandra up and running?  Is it just the size of the caches
> for the column Family + the memtable size ?

and the bloom filter and index samples from the sstable files.

Does that mean that cassandra tries to load the index and filter tables in 
memory as well, for each sstable in the keyspace?

Once the final memtable is flushed to the disk ( assuming no more writes) , does 
read path also incur the memory size of the memtable for that particular CF ?

Does cassandra try to preallocate memory after startup for each schema even if 
its not used ( not being currently written to or read from)  ? 


I apologize for so many questions,here is what I am trying to do ....
 I might need more than one schema to be configured and wondering if cassandra 
will take up memory proportional to the number of schemas "configured" as 
opposed to the ones "currently in use". This in-turn will help me decide on the 
maximum number of keyspaces that I can configure within a given heap size.

> 2) Is the size of the cache configured ( in terms of absolute numbers or
> percentages), an upper bound on the amount of memory that can be allocated
> and which grows as more data is filled up in the cache ?

no.  it's strictly the number of items you give it.  so you need to be
careful not to make it larger than you have room in the heap.

If I understand you correctly then I need to make sure that
 the sum of sizes of all items in the cache across all the keyspaces + memtable 
+ bloom filter + index samples  < Heap space

Thanks once again.
-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com


--0-1819829274-1284333001=:28922
Content-Type: text/html; charset=us-ascii

<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:arial,helvetica,sans-serif;font-size:12pt"><div>Thanks for the replies Jonathan...Couple more clarifications (in bold)<br></div><div style="font-family: arial,helvetica,sans-serif; font-size: 12pt;"><br><div style="font-family: arial,helvetica,sans-serif; font-size: 13px;"><font face="Tahoma" size="2"><hr size="1"><b><span style="font-weight: bold;">From:</span></b> Jonathan Ellis &lt;jbellis@gmail.com&gt;<br><b><span style="font-weight: bold;">To:</span></b> user@cassandra.apache.org<br><b><span style="font-weight: bold;">Sent:</span></b> Sun, September 12, 2010 1:47:09 PM<br><b><span style="font-weight: bold;">Subject:</span></b> Re: Couple of cache related questions<br></font><br>
On Sun, Sep 12, 2010 at 2:56 PM, kannan chandrasekaran<br>&lt;<a ymailto="mailto:ckannanck@yahoo.com" href="mailto:ckannanck@yahoo.com">ckannanck@yahoo.com</a>&gt; wrote:<br>&gt; 1) What determines the amount of memory used per schema ignoring the general<br>&gt; overhead to get cassandra up and running?&nbsp; Is it just the size of the caches<br>&gt; for the column Family + the memtable size ?<br><br>and the bloom filter and index samples from the sstable files.<br><br><span style="font-weight: bold;">Does that mean that cassandra tries to load the index and filter tables in memory as well, for each sstable in the keyspace</span>?<br><br><span style="font-weight: bold;">Once the final memtable is flushed to the disk ( assuming no more writes) , does read path also incur the memory size of the memtable for that particular CF ? <br><br>Does cassandra try to preallocate memory after startup for each schema even if its not used ( not being currently written
 to or read from)&nbsp; ? <br><br>I apologize for so many questions,here is what I am trying to do ....<br>&nbsp;I might need more than one schema to be configured and wondering if cassandra will take up memory proportional to the number of schemas "configured" as opposed to the ones "currently in use". This in-turn will help me decide on the maximum number of keyspaces that I can configure within a given heap size.</span><br style="font-weight: bold;"><br>&gt; 2) Is the size of the cache configured ( in terms of absolute numbers or<br>&gt; percentages), an upper bound on the amount of memory that can be allocated<br>&gt; and which grows as more data is filled up in the cache ?<br><br>no.&nbsp; it's strictly the number of items you give it.&nbsp; so you need to be<br>careful not to make it larger than you have room in the heap.<br><br style="font-weight: bold;"><span style="font-weight: bold;">If I understand you correctly then I need to make sure
 that</span><br style="font-weight: bold;"><span style="font-weight: bold;">&nbsp;the sum of sizes of all items in the cache across all the keyspaces + memtable + bloom filter + index samples&nbsp; &lt; Heap space</span><br><br><span style="font-weight: bold;">Thanks once again.</span><br>-- <br>Jonathan Ellis<br>Project Chair, Apache Cassandra<br>co-founder of Riptano, the source for professional Cassandra support<br><span><a target="_blank" href="http://riptano.com">http://riptano.com</a></span><br></div></div>
</div><br>

      </body></html>
--0-1819829274-1284333001=:28922--