incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Morton <aa...@thelastpickle.com>
Subject Re: Better to have lower or greater cardinality for partition key in CQL3?
Date Tue, 06 Aug 2013 07:51:46 GMT
> So from anyones experience, is it better to use a low cardinality
> partition key or a high cardinality.
IMHO go with whatever best supports the read paths. They all get 
If you have lots (e.g. north of 1 billion) rows per node there are extra considerations that
come into play. Cassandra 1.2 helps a lot with the bloom filters and compression meta off
heap. Basically you may need to pay more attention to memory usage at that scale. 

This is one place for the LCS can help. It allows you to have a higher bloom filter FP chance,
which results in a lower memory overhead for a given number of rows. Remember that LCS uses
roughly twice the IO though, so make sure you can handle the throughput. 

Otherwise your update workflow sounds is a perfect match for Size Tiered compaction.   

Hope that helps.

-----------------
Aaron Morton
Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 5/08/2013, at 4:57 PM, David Ward <david@shareablee.com> wrote:

> Hello,
>     Was curious what people had found to be better for
> structuring/modeling data into C*?   With my data I have two primary
> keys, one 64 bit int thats 0 - 50 million ( its unlikely to go higher
> then 70 million ever ) and another 64 bit that's probably close to
> hitting a trillion in the next year or so.   Looking at how the data
> is going to behave, for the first few months each row/record will be
> updated but after that its practically written in stone.  Still I was
> leaning toward leveled compaction as it gets updated anywhere from
> once an hour to at least once a day for the first 7 days.
> 
> So from anyones experience, is it better to use a low cardinality
> partition key or a high cardinality.   Additionally data organized by
> the low cardinality set is probably 1-6B ( and growing ) but the high
> cardinality would be 1-6MB only 2-3x a year.
> 
> 
> Thanks,
>   Dave
> 
> 
> new high cardinality keys in 1 year ~15,768,00,000
> new low cardinality keys in 1 year = 10,000-30,000
> 
> low cardinality key set size ~1-6GB
> high cardinality key set size 1-5MB


Mime
View raw message