incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: keyspace with hundreds of columnfamilies
Date Wed, 02 Jul 2014 21:43:27 GMT
The official answer, engraved in stone tablets, and carried down from the mountain: “Although
having more than dozens or hundreds of tables defined is almost certainly a Bad Idea (just
as it is a design smell in a relational database), it's relatively straightforward to allow
disabling the SlabAllocator.” Emphasis on “almost certainly a Bad Idea.”

See:
https://issues.apache.org/jira/browse/CASSANDRA-5935
“Allow disabling slab allocation”

IOW, this is considered an anti-pattern, but...

-- Jack Krupansky

From: tommaso barbugli 
Sent: Wednesday, July 2, 2014 2:16 PM
To: user@cassandra.apache.org 
Subject: Re: keyspace with hundreds of columnfamilies

Hi, 
thank you for you replies on this; regarding the arena memory is this a fixed memory allocation
or is some sort of in memory caching? I ask because I think that a substantial portion of
the column families created will not be queried that frequently (and some will become inactive
and stay like that really long time)

Thank you,
Tommaso



2014-07-02 18:35 GMT+02:00 Romain HARDOUIN <romain.hardouin@urssaf.fr>:

  Arena allocation is an improvement feature, not a limitation. 
  It was introduced in Cassandra 1.0 in order to lower memory fragmentation (and therefore
promotion failure). 
  AFAIK It's not intended to be tweaked so it might not be a good idea to change it. 

  Best, 
  Romain 

  tommaso barbugli <tbarbugli@gmail.com> a écrit sur 02/07/2014 17:40:18 :

  > De : tommaso barbugli <tbarbugli@gmail.com> 
  > A : user@cassandra.apache.org, 
  > Date : 02/07/2014 17:40 
  > Objet : Re: keyspace with hundreds of columnfamilies 

  > 
  > 1MB per column family sounds pretty bad to me; is this something I 
  > can tweak/workaround somehow? 
  > 
  > Thanks 
  > Tommaso 
  > 

  > 2014-07-02 17:21 GMT+02:00 Romain HARDOUIN <romain.hardouin@urssaf.fr>: 
  > The trap is that each CF will consume 1 MB of memory due to arena allocation. 
  > This might seem harmless but if you plan thousands of CF it means 
  > thousands of mega bytes... 
  > Up to 1,000 CF I think it could be doable, but not 10,000. 
  > 
  > Best, 
  > 
  > Romain 
  > 
  > 
  > tommaso barbugli <tbarbugli@gmail.com> a écrit sur 02/07/2014 10:13:41 :
  > 
  > > De : tommaso barbugli <tbarbugli@gmail.com> 
  > > A : user@cassandra.apache.org, 
  > > Date : 02/07/2014 10:14 
  > > Objet : keyspace with hundreds of columnfamilies 
  > > 
  > > Hi, 
  > > Are there any known issues, shortcomings about organising data in 
  > > hundreds of column families? 
  > > At this present I am running with 300 column families but I expect 
  > > that to get to a couple of thousands. 
  > > Is this something discouraged / unsupported (I am using Cassandra 2.0). 
  > > 
  > > Thanks 
  > > Tommaso

Mime
View raw message