incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From A J <s5a...@gmail.com>
Subject Re: Max # of CFs
Date Tue, 20 Mar 2012 16:12:49 GMT
I have both row cache and column cache disabled for all my CFs.

cfstats says "Bloom Filter Space Used: 1760" per CF. Assuming it is in
bytes, it is total of about 9MB of bloom filter size for 5K CFs; which
is not a lot.


On Tue, Mar 20, 2012 at 11:09 AM, Vitalii Tymchyshyn <tivv00@gmail.com> wrote:
> Hello.
>
> From my experience it's unwise to make many column families for same keys
> because you will have bloom filters and row indexes multiplied. If you have
> 5000, you should expect your heap requirements multiplied by same factor.
> Also check your cache sizes. Default AFAIR is 100000 keys per column family.
>
> 20.03.12 16:05, A J написав(ла):
>
>> ok, the last thread says that 1.0+ onwards, thousands of CFs should
>> not be a problem.
>>
>> But I am finding that all the allocated heap memory is getting consumed.
>> I started with 8GB heap and then on reading
>>
>> http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management
>> realized that minimum of 1MB per memtable is used by the per-memtable
>> arena allocator.
>> So with 5K CFs, 5GB will be used just by arena allocators.
>>
>> But even on increasing the heap to 16GB, am finding that all the heap
>> is getting consumed. Is there a different formula for heap calculation
>> when you have thousands of CFs ?
>> Any other configuration that I need to change ?
>>
>> Thanks.
>>
>> On Mon, Mar 19, 2012 at 10:35 AM, Alain RODRIGUEZ<arodrime@gmail.com>
>>  wrote:
>>>
>>> This subject was already discussed, this may help you :
>>>
>>> http://markmail.org/message/6dybhww56bxvufzf#query:+page:1+mid:6dybhww56bxvufzf+state:results
>>>
>>> If you still got questions after reading this thread or some others about
>>> the same topic, do not hesitate asking again,
>>>
>>> Alain
>>>
>>>
>>> 2012/3/19 A J<s5alye@gmail.com>
>>>>
>>>> How many Column Families are one too many for Cassandra ?
>>>> I created a db with 5000 CFs (I can go into the reasons later) but the
>>>> latency seems to be very erratic now. Not sure if it is because of the
>>>> number of CFs.
>>>>
>>>> Thanks.
>>>
>>>
>

Mime
View raw message