Are you planning to create 500,000 Super Column Families or 500,000 rows in a single Super
Column Family ?
The former is a somewhat crazy. Cassandra schemas typically have up to a few tens of Column
Families. Each column family involves a certain amount of memory overhead, this is now automatically
managed in Cassandra 0.8 (see http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/)
if I understand correctly you have 500K entities with 6k columns each. A simple first approach
to modelling this would be to use a Standard CF with a row for each entity. However the best
model is the one that serves your read requests best.
Also for background the sub columns in a super column are not indexed see http://wiki.apache.org/cassandra/CassandraLimitations
. You would probably run into this problem if you had 6000 sub columns in a super column.
Hope that helps.
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 17/08/2011, at 12:53 AM, Renato Bacelar da Silveira wrote:
> I am wondering about a certain volume situation.
>
> I currently load a Keyspace with a certain amount of SCFs.
>
> Each SCF (Super Column Family) represents an entity.
>
> Each Entity may have up to 6000 values.
>
> I am planning to have 500,000 Entities (SCF) with
> 6000 Columns (within Super Columns - number of Super Columns
> unknown), and was wondering how much resources something
> like this would require?
>
> I am struggling to have 10,000 SCF with 30 Columns (within SuperColumns),
> I get very large files, and reach a 4Gb heapspace limit very quickly on
> a single node. I use Garbage Collection where needed.
>
> Is there some secret to load 500,000 Super Column Families?
>
> Regards.
> --
> Renato da Silveira
> Senior Developer
|