incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Reek <ste...@unitedgames.com>
Subject Re: Impact of old data on performance
Date Wed, 29 Feb 2012 13:14:22 GMT
Hi Dan,

Thanks for answering.
I included the output of cfstats below.
I hope you can say something about our problems with it.

cheers,

Stefan

Keyspace: Keyspace1
     Read Count: 60703419
     Read Latency: 1.1790332096286043 ms.
     Write Count: 105871791
     Write Latency: 0.019847457393065166 ms.
     Pending Tasks: 0
         Column Family: ColumnFamily1
         SSTable count: 0
         Space used (live): 0
         Space used (total): 0
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 0
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

         Column Family: ColumnFamily2
         SSTable count: 2
         Space used (live): 482828
         Space used (total): 482828
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 3
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 9
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 0
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 258
         Compacted row maximum size: 275
         Compacted row mean size: 266

         Column Family: ColumnFamily3
         SSTable count: 0
         Space used (live): 0
         Space used (total): 0
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 0
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

         Column Family: ColumnFamily4
         SSTable count: 1
         Space used (live): 19793722
         Space used (total): 19793722
         Memtable Columns Count: 306
         Memtable Data Size: 12731
         Memtable Switch Count: 3
         Read Count: 2309
         Read Latency: NaN ms.
         Write Count: 3234
         Write Latency: 0.021 ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 465813
         Compacted row maximum size: 4784014
         Compacted row mean size: 3295196

         Column Family: ColumnFamily5
         SSTable count: 6
         Space used (live): 364775507
         Space used (total): 364775507
         Memtable Columns Count: 14193
         Memtable Data Size: 552090
         Memtable Switch Count: 3
         Read Count: 211451
         Read Latency: NaN ms.
         Write Count: 26567
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 187
         Compacted row maximum size: 333233
         Compacted row mean size: 662

         Column Family: ColumnFamily6
         SSTable count: 0
         Space used (live): 0
         Space used (total): 0
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

         Column Family: ColumnFamily7
         SSTable count: 12
         Space used (live): 6031951162
         Space used (total): 6031951162
         Memtable Columns Count: 480647
         Memtable Data Size: 29424092
         Memtable Switch Count: 43
         Read Count: 7506944
         Read Latency: 0.761 ms.
         Write Count: 3022099
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 239
         Compacted row maximum size: 93925
         Compacted row mean size: 2737

         Column Family: ColumnFamily8
         SSTable count: 1
         Space used (live): 241104
         Space used (total): 241104
         Memtable Columns Count: 2
         Memtable Data Size: 94
         Memtable Switch Count: 3
         Read Count: 685
         Read Latency: NaN ms.
         Write Count: 143
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 254
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 254
         Compacted row maximum size: 479
         Compacted row mean size: 305

         Column Family: ColumnFamily9
         SSTable count: 0
         Space used (live): 0
         Space used (total): 0
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 0
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

         Column Family: ColumnFamily10
         SSTable count: 3
         Space used (live): 22874666
         Space used (total): 22874666
         Memtable Columns Count: 11886
         Memtable Data Size: 519166
         Memtable Switch Count: 3
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 73970
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 161
         Compacted row maximum size: 103903
         Compacted row mean size: 11504

         Column Family: ColumnFamily11
         SSTable count: 3
         Space used (live): 6533310
         Space used (total): 6533310
         Memtable Columns Count: 8
         Memtable Data Size: 334
         Memtable Switch Count: 3
         Read Count: 5431
         Read Latency: NaN ms.
         Write Count: 197
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 653
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 262
         Compacted row maximum size: 17149
         Compacted row mean size: 359

         Column Family: ColumnFamily12
         SSTable count: 1
         Space used (live): 5504796
         Space used (total): 5504796
         Memtable Columns Count: 722
         Memtable Data Size: 5495928
         Memtable Switch Count: 3
         Read Count: 6413
         Read Latency: NaN ms.
         Write Count: 5784
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 167
         Compacted row maximum size: 5498089
         Compacted row mean size: 261974

         Column Family: ColumnFamily13
         SSTable count: 3
         Space used (live): 18208017
         Space used (total): 18208017
         Memtable Columns Count: 40
         Memtable Data Size: 1730
         Memtable Switch Count: 3
         Read Count: 4617
         Read Latency: NaN ms.
         Write Count: 132
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 1604
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 512
         Compacted row maximum size: 1776
         Compacted row mean size: 683

         Column Family: ColumnFamily14
         SSTable count: 9
         Space used (live): 1631088415
         Space used (total): 1631088415
         Memtable Columns Count: 355843
         Memtable Data Size: 15750980
         Memtable Switch Count: 1142
         Read Count: 37874988
         Read Latency: 0.830 ms.
         Write Count: 96285506
         Write Latency: 0.032 ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 599
         Compacted row maximum size: 1124
         Compacted row mean size: 861

         Column Family: ColumnFamily15
         SSTable count: 2
         Space used (live): 69991756
         Space used (total): 69991756
         Memtable Columns Count: 315483
         Memtable Data Size: 14827701
         Memtable Switch Count: 65
         Read Count: 5653028
         Read Latency: 6.888 ms.
         Write Count: 47666
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 160
         Compacted row maximum size: 5683855
         Compacted row mean size: 35031

         Column Family: ColumnFamily16
         SSTable count: 1
         Space used (live): 44605875
         Space used (total): 44605875
         Memtable Columns Count: 20273
         Memtable Data Size: 645383
         Memtable Switch Count: 3
         Read Count: 373155
         Read Latency: 0.058 ms.
         Write Count: 199847
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 209
         Compacted row maximum size: 82030
         Compacted row mean size: 617

         Column Family: ColumnFamily17
         SSTable count: 1
         Space used (live): 5609
         Space used (total): 5609
         Memtable Columns Count: 166
         Memtable Data Size: 2075
         Memtable Switch Count: 3
         Read Count: 17630
         Read Latency: NaN ms.
         Write Count: 542
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 522
         Compacted row maximum size: 563
         Compacted row mean size: 545

         Column Family: ColumnFamily18
         SSTable count: 3
         Space used (live): 20705514
         Space used (total): 20705514
         Memtable Columns Count: 164
         Memtable Data Size: 5892
         Memtable Switch Count: 3
         Read Count: 5605
         Read Latency: NaN ms.
         Write Count: 318
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 2314
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 521
         Compacted row maximum size: 1239
         Compacted row mean size: 604

         Column Family: ColumnFamily19
         SSTable count: 1
         Space used (live): 29516821
         Space used (total): 29516821
         Memtable Columns Count: 8402
         Memtable Data Size: 268574
         Memtable Switch Count: 3
         Read Count: 7587671
         Read Latency: 0.193 ms.
         Write Count: 72597
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 186
         Compacted row maximum size: 18443
         Compacted row mean size: 270

         Column Family: ColumnFamily20
         SSTable count: 1
         Space used (live): 711618
         Space used (total): 711618
         Memtable Columns Count: 16
         Memtable Data Size: 496
         Memtable Switch Count: 3
         Read Count: 454
         Read Latency: NaN ms.
         Write Count: 97
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 385
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 238
         Compacted row maximum size: 426
         Compacted row mean size: 303

         Column Family: ColumnFamily21
         SSTable count: 6
         Space used (live): 499913576
         Space used (total): 499913576
         Memtable Columns Count: 79019
         Memtable Data Size: 4820159
         Memtable Switch Count: 11
         Read Count: 65780
         Read Latency: NaN ms.
         Write Count: 6129833
         Write Latency: 0.036 ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 278
         Compacted row maximum size: 288
         Compacted row mean size: 284

         Column Family: ColumnFamily22
         SSTable count: 3
         Space used (live): 378024
         Space used (total): 378024
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 724
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

         Column Family: ColumnFamily23
         SSTable count: 3
         Space used (live): 11425
         Space used (total): 11425
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 1
         Read Count: 578
         Read Latency: NaN ms.
         Write Count: 1
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

         Column Family: ColumnFamily24
         SSTable count: 2
         Space used (live): 20577518
         Space used (total): 20577518
         Memtable Columns Count: 306
         Memtable Data Size: 12731
         Memtable Switch Count: 3
         Read Count: 18
         Read Latency: NaN ms.
         Write Count: 3234
         Write Latency: 0.013 ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 465334
         Compacted row maximum size: 4552398
         Compacted row mean size: 2815873

         Column Family: ColumnFamily25
         SSTable count: 1
         Space used (live): 2888
         Space used (total): 2888
         Memtable Columns Count: 2
         Memtable Data Size: 68
         Memtable Switch Count: 3
         Read Count: 1385944
         Read Latency: NaN ms.
         Write Count: 19
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache: disabled
         Row cache: disabled
         Compacted row minimum size: 223
         Compacted row maximum size: 234
         Compacted row mean size: 230

         Column Family: ColumnFamily26
         SSTable count: 0
         Space used (live): 0
         Space used (total): 0
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 0
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 200000
         Key cache size: 0
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0

----------------
Keyspace: system
     Read Count: 14
     Read Latency: 123.1865 ms.
     Write Count: 6
     Write Latency: 0.11666666666666667 ms.
     Pending Tasks: 0
         Column Family: LocationInfo
         SSTable count: 1
         Space used (live): 2871
         Space used (total): 2871
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 2
         Read Count: 2
         Read Latency: NaN ms.
         Write Count: 6
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 1
         Key cache size: 1
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 203
         Compacted row maximum size: 554
         Compacted row mean size: 378

         Column Family: HintsColumnFamily
         SSTable count: 2
         Space used (live): 4876356
         Space used (total): 4876356
         Memtable Columns Count: 0
         Memtable Data Size: 0
         Memtable Switch Count: 0
         Read Count: 12
         Read Latency: NaN ms.
         Write Count: 0
         Write Latency: NaN ms.
         Pending Tasks: 0
         Key cache capacity: 2
         Key cache size: 2
         Key cache hit rate: NaN
         Row cache: disabled
         Compacted row minimum size: 0
         Compacted row maximum size: 0
         Compacted row mean size: 0


On 02/28/2012 06:51 PM, Dan Retzlaff wrote:
> Hi Stefan. Can you share the output of nodetool cfstats?
>
> On Tue, Feb 28, 2012 at 1:50 AM, Stefan Reek <stefan@unitedgames.com 
> <mailto:stefan@unitedgames.com>> wrote:
>
>     Hi All,
>
>     We are running a 3-node cluster with Cassandra 0.6.13.
>     We are in the process of upgrading to 1.x, but can't do so for a
>     while because we can't take the cluster offline.
>     Until now 0.6.13 has run without problems, but lately we are
>     getting some performance issues.
>     We are getting timeouts on both reads and writes.
>     The cluster contains a lot of "old" data which is only read very
>     occasionally and not written to at all.
>     Is it possible for this data to negatively impact the performance
>     of the cluster?
>     If so, would it help to move this data to another
>     columnfamily/keyspace?
>
>     Cheers,
>
>     Stefan
>
>


Mime
View raw message