incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitalii Tymchyshyn <tiv...@gmail.com>
Subject Re: Max # of CFs
Date Wed, 21 Mar 2012 12:56:54 GMT
Hello.

There is also a primary row index. It's space can be controlled with 
index_interval setting. Don't know if you can look for it's memory usage 
somewhere. If I where you, I'd take jmap tool and examine heap histogram 
first, heap dump second.

Best regards, Vitalii Tymchyshyn

20.03.12 18:12, A J написав(ла):
> I have both row cache and column cache disabled for all my CFs.
>
> cfstats says "Bloom Filter Space Used: 1760" per CF. Assuming it is in
> bytes, it is total of about 9MB of bloom filter size for 5K CFs; which
> is not a lot.
>
>
> On Tue, Mar 20, 2012 at 11:09 AM, Vitalii Tymchyshyn<tivv00@gmail.com>  wrote:
>> Hello.
>>
>>  From my experience it's unwise to make many column families for same keys
>> because you will have bloom filters and row indexes multiplied. If you have
>> 5000, you should expect your heap requirements multiplied by same factor.
>> Also check your cache sizes. Default AFAIR is 100000 keys per column family.
>>
>> 20.03.12 16:05, A J написав(ла):
>>
>>> ok, the last thread says that 1.0+ onwards, thousands of CFs should
>>> not be a problem.
>>>
>>> But I am finding that all the allocated heap memory is getting consumed.
>>> I started with 8GB heap and then on reading
>>>
>>> http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management
>>> realized that minimum of 1MB per memtable is used by the per-memtable
>>> arena allocator.
>>> So with 5K CFs, 5GB will be used just by arena allocators.
>>>
>>> But even on increasing the heap to 16GB, am finding that all the heap
>>> is getting consumed. Is there a different formula for heap calculation
>>> when you have thousands of CFs ?
>>> Any other configuration that I need to change ?
>>>
>>> Thanks.
>>>
>>> On Mon, Mar 19, 2012 at 10:35 AM, Alain RODRIGUEZ<arodrime@gmail.com>
>>>   wrote:
>>>> This subject was already discussed, this may help you :
>>>>
>>>> http://markmail.org/message/6dybhww56bxvufzf#query:+page:1+mid:6dybhww56bxvufzf+state:results
>>>>
>>>> If you still got questions after reading this thread or some others about
>>>> the same topic, do not hesitate asking again,
>>>>
>>>> Alain
>>>>
>>>>
>>>> 2012/3/19 A J<s5alye@gmail.com>
>>>>> How many Column Families are one too many for Cassandra ?
>>>>> I created a db with 5000 CFs (I can go into the reasons later) but the
>>>>> latency seems to be very erratic now. Not sure if it is because of the
>>>>> number of CFs.
>>>>>
>>>>> Thanks.
>>>>


Mime
View raw message