incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean" <>
Subject Re: Data Model - Additional Column Families or one CF?
Date Tue, 26 Feb 2013 15:27:40 GMT
The bottleneck is RAM, each CF uses more RAM.  We tried to go above 15000 column families and
that hurt big time so we added a feature to PlayOrm and now have 60,000 virtual Column families
all in one column family.  This turned out to be HUGE benefit though as those 60,000 tables
now have been easy to modify cluster settings in one shot.  Right now, we are modifying the
false positive ratio to free up ram on just one column family that all 60,000 run in.

You can always follow the same pattern PlayOrm uses which is just to prefix every row key
but we preferred something else do the heavy lifting for us (ie. PlayOrm), plus the command
line tool isn't too bad for inspecting the data with PlayOrm either.  Of course, we now have
7 billion rows and that is a new issue for us.  While we could just add more nodes and scale,
they are pushing us to make it even more cost effective so we are trying to scale on a single
node first then scale out.


From: Javier Sotelo <<>>
Reply-To: "<>" <<>>
Date: Tuesday, February 26, 2013 12:27 AM
To: "<>" <<>>
Subject: Re: Data Model - Additional Column Families or one CF?


Would 50 CFs be pushing it? According to,
"This has been tested to work across hundreds or even thousands of ColumnFamilies."

What is the bottleneck, IO?


On Sun, Feb 24, 2013 at 5:51 PM, Adam Venturella <<>>

Thanks Aaron, this was a big help!

Sent from Mailbox<> for iPhone

On Thu, Feb 21, 2013 at 9:27 AM, aaron morton <<>>

If you have a limited / known number (say < 30)  of types, I would create a CF for each
of them.

If the number of types is unknown or very large I would have one CF with the row key you described.

Generally I avoid data models that require new CF's as the data grows. Additionally having
different CF's allows you to use different cache settings, compactions settings and even storage


Aaron Morton
Freelance Cassandra Developer
New Zealand


On 21/02/2013, at 7:43 AM, Adam Venturella <<>>

My data needs only require me to store JSON, and I can handle this in 1 column family by prefixing
row keys with a type, for example:


Where comments: represents the prefix and {message_id} represents some row key to a message
object in the same column family.

In this case comments:{message_id} would be a wide row using comment creation time and descending
clustering order to sort the messages as they are added.

My question is, would I be better off splitting comments into their own Column Family or is
storing them in with the Messages Column Family sufficient, they are all messages after all.

Or do Column Families really just provide a nice organizational front for data. I'm just storing

View raw message