Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 791B49F10 for ; Fri, 7 Oct 2011 14:47:29 +0000 (UTC) Received: (qmail 59041 invoked by uid 500); 7 Oct 2011 14:47:27 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 59013 invoked by uid 500); 7 Oct 2011 14:47:27 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 59005 invoked by uid 99); 7 Oct 2011 14:47:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Oct 2011 14:47:27 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [74.125.82.172] (HELO mail-wy0-f172.google.com) (74.125.82.172) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Oct 2011 14:47:21 +0000 Received: by wyg24 with SMTP id 24so987256wyg.31 for ; Fri, 07 Oct 2011 07:47:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.4.221 with SMTP id 71mr2839776wej.3.1317998820906; Fri, 07 Oct 2011 07:47:00 -0700 (PDT) Received: by 10.216.153.195 with HTTP; Fri, 7 Oct 2011 07:47:00 -0700 (PDT) In-Reply-To: References: Date: Fri, 7 Oct 2011 09:47:00 -0500 Message-ID: Subject: Re: For multi-tenant, is it good to have a key space for each tenant? From: David McNelis To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001485f8606e88d7f004aeb68338 X-Virus-Checked: Checked by ClamAV on apache.org --001485f8606e88d7f004aeb68338 Content-Type: text/plain; charset=ISO-8859-1 So at the end of the day its going to be limited by available memory then? Going by this line: Do note that a minimum of 1MB per memtable is used by the per-memtable arena allocator also introduced in 1.0, which is worth keeping in mind if you are looking at going from thousands to tens of thousands of ColumnFamilies. Then you'd be looking at a requirement of 1gig memory for each 1024 column families had on your cluster, regardless of number of keyspaces those lived in. So in order to have 10s of thousands of CFs, one would need 10s of gigs of ram on each node just to handle that overhead...at least as of v1. On Fri, Oct 7, 2011 at 9:40 AM, Jonathan Ellis wrote: > On Fri, Oct 7, 2011 at 9:36 AM, David McNelis > wrote: > > In some documentation I've read it says that > > keyspace's take up the majority of the resources > > This has never been the case. > > > in a couple of older > > threads they talked about getting the number of column families down. > > This was good advice pre-0.8. > > I covered the state of 0.8 and 1.0 here: > > http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of DataStax, the source for professional Cassandra support > http://www.datastax.com > -- *David McNelis* Lead Software Engineer Agentis Energy www.agentisenergy.com o: 630.359.6395 c: 219.384.5143 *A Smart Grid technology company focused on helping consumers of energy control an often under-managed resource.* --001485f8606e88d7f004aeb68338 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable So at the end of the day its going to be limited by available memory then? = =A0Going by this line:

Do note that a minimum of 1MB per memtable is used by the=A0per-memtable arena = allocator=A0also introduced in 1.0, which is worth keeping in mind if y= ou are looking at going from thousands to tens of thousands of ColumnFamili= es.

=A0=A0Then you'd be looking at a req= uirement of 1gig =A0memory for each =A01024 column families had on your clu= ster, regardless of number of keyspaces those lived in. =A0So in order to h= ave 10s of thousands of CFs, one would need 10s of gigs of ram on each node= just to handle that overhead...at least as =A0of =A0v1. =A0<= span class=3D"Apple-style-span" style=3D"line-height: 20px;">

On Fri, Oct 7, 2011 at 9:40 AM= , Jonathan Ellis <jbellis@gmail.com> wrote:
On Fri, Oct 7, 2011 at 9:36 AM, David McNelis
<dmcnelis@agentisenergy.co= m> wrote:
>=A0In some documentation I've read it says that
> keyspace's take up the majority of the resources

This has never been the case.

> in a couple of older
> threads they talked about getting the number of column families down.<= br>
This was good advice pre-0.8.

I covered the state of 0.8 and 1.0 here:
http://www.datasta= x.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-ma= nagement

--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.c= om



--
Da= vid McNelis
Lead Software Engine= er
Agentis Energy
o: 630.359.6395
c: 219.384.5143<= /span>

A Smart Grid technology company focused on helping consumers of = energy control an often under-managed resource.


--001485f8606e88d7f004aeb68338--