From user-return-25276-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Tue Apr 3 01:49:59 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 859209F9E for ; Tue, 3 Apr 2012 01:49:59 +0000 (UTC) Received: (qmail 86576 invoked by uid 500); 3 Apr 2012 01:49:57 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 86558 invoked by uid 500); 3 Apr 2012 01:49:57 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 86549 invoked by uid 99); 3 Apr 2012 01:49:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2012 01:49:57 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a56.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2012 01:49:50 +0000 Received: from homiemail-a56.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a56.g.dreamhost.com (Postfix) with ESMTP id 7316AFE059 for ; Mon, 2 Apr 2012 18:49:28 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=lMj02jt9mG vkiz5LV35LQE+JhmrttVzUw2G/ToMcgU/1SDuLA24noYK/LwBWpo4BXi0cLr0K0T UyL/BSq8cNpB3VGZ4l/9cN0aId92bO+W7HeiWaLfXAm2H3++jwp81Jrz0JX/esUr xr3Vf1NVHqhLEmhTDsURaMPx1T04vTLbY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=lEK1569CZHkEEPaF hx7FABsgpcU=; b=1ialYgpzhH5vxD0vvN6q+Q7UxkG9frINXC/lEM5CBK/rcvvo Odn7dIgOSOaqtYKgqCNbYj0yaGrh3t5sCG4+1z+BodSdO85cg405u/A0MXu6LpCn ccOCGdnW50fy92p/H11HFHPVGFnFrmVbbpB8aLy6D4P1tiaCM7cA5/ZiDwY= Received: from [10.100.150.56] (unknown [198.48.2.2]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a56.g.dreamhost.com (Postfix) with ESMTPSA id ECCF9FE00C for ; Mon, 2 Apr 2012 18:49:27 -0700 (PDT) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: multipart/alternative; boundary="Apple-Mail=_9C8F43C2-93F6-4F9F-B17A-CBFEE27429B4" Subject: Re: key cache size calculation Date: Tue, 3 Apr 2012 13:49:25 +1200 In-Reply-To: To: user@cassandra.apache.org References: Message-Id: X-Mailer: Apple Mail (2.1257) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_9C8F43C2-93F6-4F9F-B17A-CBFEE27429B4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Take a look at the key cache hit rate in nodetool cfstats.=20 One approach is to increase the cache size until you do not see a = matching increase in the hit rate.=20 > Is there a limit to key cache size? I know that is all taken from heap = but how much max we can go with setting the key cache sizes? It's pretty much a memory thing.=20 Each entry maps a description of the SStable and the key to the offset = in the index file. (off the top of my head) The SSTable description is = shared, the row key, row token (16 bytes) and offset (offset) will take = up space. Cheers =20 ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 3/04/2012, at 11:24 AM, Shoaib Mir wrote: > Hi guys, >=20 > We are calculating key cache size right now. There is this column = family with ~ 100 million columns and right now we have the cache size = set at 2 million. >=20 > I suspect that the active data we got is not all fitting in the 2 = million cache size and we at times are getting query execution time way = higher then the normal. Is there a limit to key cache size? I know that = is all taken from heap but how much max we can go with setting the key = cache sizes? >=20 > cheers, > Shoaib --Apple-Mail=_9C8F43C2-93F6-4F9F-B17A-CBFEE27429B4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 Take = a look at the key cache hit rate in nodetool = cfstats. 

One approach is to increase the cache = size until you do not see a matching increase in the hit = rate. 

Is there = a limit to key cache size? I know that is all taken from heap but how = much max we can go with setting the key cache = sizes?
It's pretty much a memory = thing. 

Each entry maps a description of = the SStable and the key to the offset in the index file. (off the top of = my head) The SSTable description is shared, the row key, row token (16 = bytes) and offset (offset) will take up = space.

Cheers
 
http://www.thelastpickle.com

On 3/04/2012, at 11:24 AM, Shoaib Mir wrote:

Hi = guys,

We are calculating key cache size right now. There is this = column family with ~ 100 million columns and right now we have the cache = size set at 2 million.

I suspect that the active data we got is = not all fitting in the 2 million cache size and we at times are getting = query execution time way higher then the normal. Is there a limit to key = cache size? I know that is all taken from heap but how much max we can = go with setting the key cache sizes?

cheers,
Shoaib

= --Apple-Mail=_9C8F43C2-93F6-4F9F-B17A-CBFEE27429B4--