incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitalii Tymchyshyn <>
Subject Re: Max # of CFs
Date Wed, 21 Mar 2012 12:56:54 GMT

There is also a primary row index. It's space can be controlled with 
index_interval setting. Don't know if you can look for it's memory usage 
somewhere. If I where you, I'd take jmap tool and examine heap histogram 
first, heap dump second.

Best regards, Vitalii Tymchyshyn

20.03.12 18:12, A J написав(ла):
> I have both row cache and column cache disabled for all my CFs.
> cfstats says "Bloom Filter Space Used: 1760" per CF. Assuming it is in
> bytes, it is total of about 9MB of bloom filter size for 5K CFs; which
> is not a lot.
> On Tue, Mar 20, 2012 at 11:09 AM, Vitalii Tymchyshyn<>  wrote:
>> Hello.
>>  From my experience it's unwise to make many column families for same keys
>> because you will have bloom filters and row indexes multiplied. If you have
>> 5000, you should expect your heap requirements multiplied by same factor.
>> Also check your cache sizes. Default AFAIR is 100000 keys per column family.
>> 20.03.12 16:05, A J написав(ла):
>>> ok, the last thread says that 1.0+ onwards, thousands of CFs should
>>> not be a problem.
>>> But I am finding that all the allocated heap memory is getting consumed.
>>> I started with 8GB heap and then on reading
>>> realized that minimum of 1MB per memtable is used by the per-memtable
>>> arena allocator.
>>> So with 5K CFs, 5GB will be used just by arena allocators.
>>> But even on increasing the heap to 16GB, am finding that all the heap
>>> is getting consumed. Is there a different formula for heap calculation
>>> when you have thousands of CFs ?
>>> Any other configuration that I need to change ?
>>> Thanks.
>>> On Mon, Mar 19, 2012 at 10:35 AM, Alain RODRIGUEZ<>
>>>   wrote:
>>>> This subject was already discussed, this may help you :
>>>> If you still got questions after reading this thread or some others about
>>>> the same topic, do not hesitate asking again,
>>>> Alain
>>>> 2012/3/19 A J<>
>>>>> How many Column Families are one too many for Cassandra ?
>>>>> I created a db with 5000 CFs (I can go into the reasons later) but the
>>>>> latency seems to be very erratic now. Not sure if it is because of the
>>>>> number of CFs.
>>>>> Thanks.

View raw message