incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eran Chinthaka Withana <eran.chinth...@gmail.com>
Subject Re: Key cache hit rate issue
Date Thu, 16 Feb 2012 22:46:00 GMT
Hi Todd,

Thanks for the reply. But I don't think the settings that you mentioned is
not playing any role as those are set to 0.85 and 0.6 in my cassandra.yaml
and the proportion between the number I see as space used and the amount I
set is much less than those numbers.

Thanks,
Eran Chinthaka Withana


On Thu, Feb 16, 2012 at 2:18 PM, Todd Burruss <bburruss@expedia.com> wrote:

> there is a setting in the yaml file that helps relieve memory pressure by
> reducing the row cache.  it is based on the percent of memory used by the
> JVM
>
> the setting are, reduce_cache_sizes_at and reduce_cache_capacity_to.  see
> how much free memory you have and if the numbers suggest that you have hit
> that limit, therefore reducing row cache size
>
>
>
> From: Eran Chinthaka Withana <eran.chinthaka@gmail.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Thu, 16 Feb 2012 13:52:38 -0800
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: Key cache hit rate issue
>
> Hi Jonathan,
>
> Thanks for the reply. Yes there is a possibility that the keys can be
> distributed in multiple SSTables, but my data access patterns are such that
> I always read/write the whole row. So I expect all the data to be in the
> same SSTable (please correct me if I'm wrong).
>
> For some reason 16637958 (the keys cached) has become a golden number and
> I don't see key cache increasing beyond that. I also checked memory and I
> have about 4GB left in JVM memory and didn't see any issues on logs.
>
> Thanks,
> Eran Chinthaka Withana
>
>
> On Thu, Feb 16, 2012 at 1:20 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>
>> So, you have roughly 1/6 of your (physical) row keys cached and about
>> 1/4 cache hit rate, which doesn't sound unreasonable to me.  Remember,
>> each logical key may be spread across multiple physical sstables --
>> each (key, sstable) pair is one entry in the key cache.
>>
>> On Thu, Feb 16, 2012 at 1:48 PM, Eran Chinthaka Withana
>> <eran.chinthaka@gmail.com> wrote:
>> > Hi Aaron,
>> >
>> > Here it is.
>> >
>> > Keyspace: XXXX
>> > Read Count: 1123637972
>> > Read Latency: 5.757938114343114 ms.
>> > Write Count: 128201833
>> > Write Latency: 0.0682576607387509 ms.
>> > Pending Tasks: 0
>> > Column Family: YY
>> > SSTable count: 18
>> > Space used (live): 103318720685
>> > Space used (total): 103318720685
>> > Number of Keys (estimate): 92404992
>> > Memtable Columns Count: 1425580
>> > Memtable Data Size: 359655747
>> > Memtable Switch Count: 2522
>> > Read Count: 1123637972
>> > Read Latency: 14.731 ms.
>> > Write Count: 128201833
>> > Write Latency: NaN ms.
>> > Pending Tasks: 0
>> > Bloom Filter False Postives: 1488
>> > Bloom Filter False Ratio: 0.00000
>> > Bloom Filter Space Used: 331522920
>> > Key cache capacity: 16637958
>> > Key cache size: 16637958
>> > Key cache hit rate: 0.2708333333333333
>> > Row cache: disabled
>> > Compacted row minimum size: 51
>> > Compacted row maximum size: 6866
>> > Compacted row mean size: 2560
>> >
>> > Thanks,
>> > Eran Chinthaka Withana
>> >
>> >
>> >
>> > On Thu, Feb 16, 2012 at 12:30 AM, aaron morton <aaron@thelastpickle.com
>> >
>> > wrote:
>> >>
>> >> Its in the order of 261 to 8000 and the ratio is 0.00. But i guess
>> 8000 is
>> >> bit high. Is there a way to fix/improve it?
>> >>
>> >> Sorry I don't understand what you mean. But if the ratio is 0.0 all is
>> >> good.
>> >>
>> >> Could you include the full output from cfstats for the CF you are
>> looking
>> >> at ?
>> >>
>> >> Cheers
>> >>
>> >> -----------------
>> >> Aaron Morton
>> >> Freelance Developer
>> >> @aaronmorton
>> >> http://www.thelastpickle.com
>> >>
>> >> On 15/02/2012, at 1:00 PM, Eran Chinthaka Withana wrote:
>> >>
>> >> Its in the order of 261 to 8000 and the ratio is 0.00. But i guess
>> 8000 is
>> >> bit high. Is there a way to fix/improve it?
>> >>
>> >> Thanks,
>> >> Eran Chinthaka Withana
>> >>
>> >>
>> >> On Tue, Feb 14, 2012 at 3:42 PM, aaron morton <aaron@thelastpickle.com
>> >
>> >> wrote:
>> >>>
>> >>> Out of interest what does cfstats say about the bloom filter stats ?
A
>> >>> high false positive could lead to a low key cache hit rate.
>> >>>
>> >>> Also, is there a way to warm start the key cache, meaning pre-load the
>> >>> amount of keys I set as keys_cached?
>> >>>
>> >>> See key_cache_save_period when creating the CF.
>> >>>
>> >>> Cheers
>> >>>
>> >>>
>> >>> -----------------
>> >>> Aaron Morton
>> >>> Freelance Developer
>> >>> @aaronmorton
>> >>> http://www.thelastpickle.com
>> >>>
>> >>> On 15/02/2012, at 5:54 AM, Eran Chinthaka Withana wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> I'm using Cassandra 1.0.7 and I've set the keys_cached to about 80%
>> >>> (using the numerical values). This is visible in cfstats too. But I'm
>> >>> getting less than 20% (or sometimes even 0%) key cache hit rate.
>> Well, the
>> >>> data access pattern is not the issue here as I know they are
>> retrieving the
>> >>> same row multiple times. I'm using hector client with dynamic load
>> balancing
>> >>> policy with consistency ONE for both reads and writes. Any ideas on
>> how to
>> >>> find the issue and fix this?
>> >>>
>> >>> Here is what I see on cfstats.
>> >>>
>> >>> Key cache capacity: 16637958
>> >>> Key cache size: 16637958
>> >>> Key cache hit rate: 0.045454545454545456
>> >>>
>> >>> Also, is there a way to warm start the key cache, meaning pre-load the
>> >>> amount of keys I set as keys_cached?
>> >>>
>> >>> Thanks,
>> >>> Eran
>> >>>
>> >>>
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>
>
>

Mime
View raw message