cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daning <dan...@netseer.com>
Subject Re: Key count mismatch in cluster for a column family
Date Mon, 07 Nov 2011 18:15:59 GMT
Sylvain - We have similar problem but the discrepancy is not that big. 
Do we have to do major compaction to fix it?  We did not do 'nodetool 
compact', just did repair regularly, which triggers minor compaction.

Thanks,

Daning

On 10/26/2011 03:23 AM, Sylvain Lebresne wrote:
> The estimate for the number of keys is computed by summing the key
> estimate for each sstable of the CF. For each sstable, the estimate
> should be fairly good. However, that's when we sum all the sstable estimates
> that we can loose potentially a lot of precision if there is a lot of rows that
> have parts in different sstables. But that in turn would suggest a problem
> with compaction lacking badly behind, especially with leveled compaction.
>
> --
> Sylvain
>
> On Wed, Oct 26, 2011 at 3:58 AM, Terry Cumaranatunge<cumarana@gmail.com>  wrote:
>> I have a cluster of 8 nodes all running 1.0. The stats shown on the 1st node
>> on one of the CFs for the number of keys is much larger than expected. The
>> first node shows the key count estimate to be 9.2M whereas the rest report
>> ~650K on each node. The 650K is in the correct neighborhood of the number of
>> keys that have been inserted. The counts are comparable for all other CFs
>> across the cluster. I'm using Level compaction, but no compression.
>>
>> The 'nodetool ring' shows that the load is equal across all nodes. What
>> could cause this large disparity in the number of keys? Is this just a stats
>> issue or does this suggest a functional problem?
>>
>> 1st node:
>>          Column Family: uid
>>                  SSTable count: 395
>>                  Space used (live): 13752624444
>>                  Space used (total): 5482088532
>>                  Number of Keys (estimate): 9215104
>>                  Memtable Columns Count: 514952
>>                  Memtable Data Size: 295213448
>>                  Memtable Switch Count: 290
>>                  Read Count: 193102511
>>                  Read Latency: 0.146 ms.
>>                  Write Count: 176934874
>>                  Write Latency: 0.018 ms.
>>                  Pending Tasks: 0
>>                  Key cache capacity: 8302131
>>                  Key cache size: 8302131
>>                  Key cache hit rate: 0.8644664668071792
>>                  Row cache: disabled
>>                  Compacted row minimum size: 87
>>                  Compacted row maximum size: 7007506
>>                  Compacted row mean size: 8944
>> 2nd node:
>>          Column Family: uid
>>                  SSTable count: 402
>>                  Space used (live): 13723958304
>>                  Space used (total): 4044833220
>>                  Number of Keys (estimate): 652928
>>                  Memtable Columns Count: 170290
>>                  Memtable Data Size: 102378904
>>                  Memtable Switch Count: 272
>>                  Read Count: 192463595
>>                  Read Latency: 0.289 ms.
>>                  Write Count: 176527238
>>                  Write Latency: 0.014 ms.
>>                  Pending Tasks: 0
>>                  Key cache capacity: 8783058
>>                  Key cache size: 8783058
>>                  Key cache hit rate: 0.7865727464740025
>>                  Row cache: disabled
>>                  Compacted row minimum size: 87
>>                  Compacted row maximum size: 7007506
>>                  Compacted row mean size: 12151
>> 3rd node:
>>                Column Family: uid
>>                  SSTable count: 401
>>                  Space used (live): 13204714872
>>                  Space used (total): 4030024144
>>                  Number of Keys (estimate): 675968
>>                  Memtable Columns Count: 42881
>>                  Memtable Data Size: 30992298
>>                  Memtable Switch Count: 304
>>                  Read Count: 190769879
>>                  Read Latency: 0.224 ms.
>>                  Write Count: 175381826
>>                  Write Latency: 0.014 ms.
>>                  Pending Tasks: 0
>>                  Key cache capacity: 8920108
>>                  Key cache size: 8920108
>>                  Key cache hit rate: 0.8053563128870577
>>                  Row cache: disabled
>>                  Compacted row minimum size: 87
>>                  Compacted row maximum size: 4866323
>>                  Compacted row mean size: 12074
>>


Mime
View raw message