incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Terry Cumaranatunge <cumar...@gmail.com>
Subject Key count mismatch in cluster for a column family
Date Wed, 26 Oct 2011 01:58:24 GMT
I have a cluster of 8 nodes all running 1.0. The stats shown on the 1st node
on one of the CFs for the number of keys is much larger than expected. The
first node shows the key count estimate to be 9.2M whereas the rest report
~650K on each node. The 650K is in the correct neighborhood of the number of
keys that have been inserted. The counts are comparable for all other CFs
across the cluster. I'm using Level compaction, but no compression.

The 'nodetool ring' shows that the load is equal across all nodes. What
could cause this large disparity in the number of keys? Is this just a stats
issue or does this suggest a functional problem?

1st node:
        Column Family: uid
                SSTable count: 395
                Space used (live): 13752624444
                Space used (total): 5482088532
                Number of Keys (estimate): 9215104
                Memtable Columns Count: 514952
                Memtable Data Size: 295213448
                Memtable Switch Count: 290
                Read Count: 193102511
                Read Latency: 0.146 ms.
                Write Count: 176934874
                Write Latency: 0.018 ms.
                Pending Tasks: 0
                Key cache capacity: 8302131
                Key cache size: 8302131
                Key cache hit rate: 0.8644664668071792
                Row cache: disabled
                Compacted row minimum size: 87
                Compacted row maximum size: 7007506
                Compacted row mean size: 8944
2nd node:
        Column Family: uid
                SSTable count: 402
                Space used (live): 13723958304
                Space used (total): 4044833220
                Number of Keys (estimate): 652928
                Memtable Columns Count: 170290
                Memtable Data Size: 102378904
                Memtable Switch Count: 272
                Read Count: 192463595
                Read Latency: 0.289 ms.
                Write Count: 176527238
                Write Latency: 0.014 ms.
                Pending Tasks: 0
                Key cache capacity: 8783058
                Key cache size: 8783058
                Key cache hit rate: 0.7865727464740025
                Row cache: disabled
                Compacted row minimum size: 87
                Compacted row maximum size: 7007506
                Compacted row mean size: 12151
3rd node:
              Column Family: uid
                SSTable count: 401
                Space used (live): 13204714872
                Space used (total): 4030024144
                Number of Keys (estimate): 675968
                Memtable Columns Count: 42881
                Memtable Data Size: 30992298
                Memtable Switch Count: 304
                Read Count: 190769879
                Read Latency: 0.224 ms.
                Write Count: 175381826
                Write Latency: 0.014 ms.
                Pending Tasks: 0
                Key cache capacity: 8920108
                Key cache size: 8920108
                Key cache hit rate: 0.8053563128870577
                Row cache: disabled
                Compacted row minimum size: 87
                Compacted row maximum size: 4866323
                Compacted row mean size: 12074

Mime
View raw message