Return-Path: 
 <user-return-24469-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 2FC6D9CFC
	for <apmail-cassandra-user-archive@www.apache.org>;
 Wed, 29 Feb 2012 18:49:30 +0000 (UTC)
Received: (qmail 7601 invoked by uid 500); 29 Feb 2012 18:49:27 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 7578 invoked by uid 500); 29 Feb 2012 18:49:27 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 7569 invoked by uid 99); 29 Feb 2012 18:49:27 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Feb 2012 18:49:27 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of dretzlaff@gmail.com designates
 74.125.82.172 as permitted sender)
Received: from [74.125.82.172] (HELO mail-we0-f172.google.com) (74.125.82.172)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Feb 2012 18:49:19 +0000
Received: by werb10 with SMTP id b10so3420841wer.31
        for <user@cassandra.apache.org>; Wed, 29 Feb 2012 10:48:59 -0800 (PST)
Received-SPF: pass (google.com: domain of dretzlaff@gmail.com designates
 10.180.95.34 as permitted sender) client-ip=10.180.95.34;
Authentication-Results: mr.google.com;
 spf=pass (google.com: domain of dretzlaff@gmail.com designates 10.180.95.34
 as permitted sender) smtp.mail=dretzlaff@gmail.com;
 dkim=pass header.i=dretzlaff@gmail.com
Received: from mr.google.com ([10.180.95.34])
        by 10.180.95.34 with SMTP id dh2mr3386847wib.15.1330541339690
 (num_hops = 1);
        Wed, 29 Feb 2012 10:48:59 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=Xp8b43XLQF7QAqesRk6WGfqGG7wnV6Bf+IlwYlqFfpU=;
        b=Z65NddqbOeCes+3GRhB7wx4VAH7D9PKEXS2Te1KGzt4MZnMyNTKRBNN13WFXNt0ikh
         sAK7CtrKqKYfDHe+riKkqM0NpbiQafTtYdFa/MN6aaB/G0JbU66/cP5lSOraDSz5zFnz
         AW9vbFne2nMWGSdXdrOm/OLfSXyElZ+y3qVkk=
MIME-Version: 1.0
Received: by 10.180.95.34 with SMTP id dh2mr2721489wib.15.1330541339613; Wed,
 29 Feb 2012 10:48:59 -0800 (PST)
Received: by 10.216.166.134 with HTTP; Wed, 29 Feb 2012 10:48:59 -0800 (PST)
In-Reply-To: <4F4E24AE.8080702@unitedgames.com>
References: <4F4CA35F.3040807@unitedgames.com>
	<CAH0XANbHBtM6EiBo1Ln+_7=G6_KX_BmFgJ1VFscCmN6KPvzCqw@mail.gmail.com>
	<4F4E24AE.8080702@unitedgames.com>
Date: Wed, 29 Feb 2012 10:48:59 -0800
Message-ID: 
 <CAH0XANYbE175mB1QBC8Vn+-Re0pph+61D8gPiYjV=Qr4-ekpKg@mail.gmail.com>
Subject: Re: Impact of old data on performance
From: Dan Retzlaff <dretzlaff@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=f46d04448035e8198404ba1ecb03
X-Virus-Checked: Checked by ClamAV on apache.org

--f46d04448035e8198404ba1ecb03
Content-Type: text/plain; charset=ISO-8859-1

First to be clear, I'm not an expert but I suggested "cfstats" because it
can show unhealthy signs. That said, yours looks okay to me: few live
SSTables per column family, reasonable quantity of data... The next things
I'd verify are that (1) the JVM isn't swapping out during these periods of
bad performance, (2) GC completes in less than your RPC timeout (check the
logs), and (3) your internode network connection isn't degraded by another
application.

The fact that *writes* are timing out is interesting, since existing data
should have little to no effect on those.

BTW, since ColumnFamily7 sees most of your action, you might consider
whether a key/row cache is appropriate. By my count it has 2M rows and 7M
reads since startup. But this fine-tuning is probably a distraction from
whatever's causing your timeouts.

Hope that helps,
Dan

On Wed, Feb 29, 2012 at 5:14 AM, Stefan Reek <stefan@unitedgames.com> wrote:

> **
> Hi Dan,
>
> Thanks for answering.
> I included the output of cfstats below.
> I hope you can say something about our problems with it.
>
> cheers,
>
> Stefan
>
> Keyspace: Keyspace1
>     Read Count: 60703419
>     Read Latency: 1.1790332096286043 ms.
>     Write Count: 105871791
>     Write Latency: 0.019847457393065166 ms.
>     Pending Tasks: 0
>         Column Family: ColumnFamily1
>         SSTable count: 0
>         Space used (live): 0
>         Space used (total): 0
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 0
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>         Column Family: ColumnFamily2
>         SSTable count: 2
>         Space used (live): 482828
>         Space used (total): 482828
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 3
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 9
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 0
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 258
>         Compacted row maximum size: 275
>         Compacted row mean size: 266
>
>         Column Family: ColumnFamily3
>         SSTable count: 0
>         Space used (live): 0
>         Space used (total): 0
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 0
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>         Column Family: ColumnFamily4
>         SSTable count: 1
>         Space used (live): 19793722
>         Space used (total): 19793722
>         Memtable Columns Count: 306
>         Memtable Data Size: 12731
>         Memtable Switch Count: 3
>         Read Count: 2309
>         Read Latency: NaN ms.
>         Write Count: 3234
>         Write Latency: 0.021 ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 465813
>         Compacted row maximum size: 4784014
>         Compacted row mean size: 3295196
>
>         Column Family: ColumnFamily5
>         SSTable count: 6
>         Space used (live): 364775507
>         Space used (total): 364775507
>         Memtable Columns Count: 14193
>         Memtable Data Size: 552090
>         Memtable Switch Count: 3
>         Read Count: 211451
>         Read Latency: NaN ms.
>         Write Count: 26567
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 187
>         Compacted row maximum size: 333233
>         Compacted row mean size: 662
>
>         Column Family: ColumnFamily6
>         SSTable count: 0
>         Space used (live): 0
>         Space used (total): 0
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>         Column Family: ColumnFamily7
>         SSTable count: 12
>         Space used (live): 6031951162
>         Space used (total): 6031951162
>         Memtable Columns Count: 480647
>         Memtable Data Size: 29424092
>         Memtable Switch Count: 43
>         Read Count: 7506944
>         Read Latency: 0.761 ms.
>         Write Count: 3022099
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 239
>         Compacted row maximum size: 93925
>         Compacted row mean size: 2737
>
>         Column Family: ColumnFamily8
>         SSTable count: 1
>         Space used (live): 241104
>         Space used (total): 241104
>         Memtable Columns Count: 2
>         Memtable Data Size: 94
>         Memtable Switch Count: 3
>         Read Count: 685
>         Read Latency: NaN ms.
>         Write Count: 143
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 254
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 254
>         Compacted row maximum size: 479
>         Compacted row mean size: 305
>
>         Column Family: ColumnFamily9
>         SSTable count: 0
>         Space used (live): 0
>         Space used (total): 0
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 0
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>         Column Family: ColumnFamily10
>         SSTable count: 3
>         Space used (live): 22874666
>         Space used (total): 22874666
>         Memtable Columns Count: 11886
>         Memtable Data Size: 519166
>         Memtable Switch Count: 3
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 73970
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 161
>         Compacted row maximum size: 103903
>         Compacted row mean size: 11504
>
>         Column Family: ColumnFamily11
>         SSTable count: 3
>         Space used (live): 6533310
>         Space used (total): 6533310
>         Memtable Columns Count: 8
>         Memtable Data Size: 334
>         Memtable Switch Count: 3
>         Read Count: 5431
>         Read Latency: NaN ms.
>         Write Count: 197
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 653
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 262
>         Compacted row maximum size: 17149
>         Compacted row mean size: 359
>
>         Column Family: ColumnFamily12
>         SSTable count: 1
>         Space used (live): 5504796
>         Space used (total): 5504796
>         Memtable Columns Count: 722
>         Memtable Data Size: 5495928
>         Memtable Switch Count: 3
>         Read Count: 6413
>         Read Latency: NaN ms.
>         Write Count: 5784
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 167
>         Compacted row maximum size: 5498089
>         Compacted row mean size: 261974
>
>         Column Family: ColumnFamily13
>         SSTable count: 3
>         Space used (live): 18208017
>         Space used (total): 18208017
>         Memtable Columns Count: 40
>         Memtable Data Size: 1730
>         Memtable Switch Count: 3
>         Read Count: 4617
>         Read Latency: NaN ms.
>         Write Count: 132
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 1604
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 512
>         Compacted row maximum size: 1776
>         Compacted row mean size: 683
>
>         Column Family: ColumnFamily14
>         SSTable count: 9
>         Space used (live): 1631088415
>         Space used (total): 1631088415
>         Memtable Columns Count: 355843
>         Memtable Data Size: 15750980
>         Memtable Switch Count: 1142
>         Read Count: 37874988
>         Read Latency: 0.830 ms.
>         Write Count: 96285506
>         Write Latency: 0.032 ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 599
>         Compacted row maximum size: 1124
>         Compacted row mean size: 861
>
>         Column Family: ColumnFamily15
>         SSTable count: 2
>         Space used (live): 69991756
>         Space used (total): 69991756
>         Memtable Columns Count: 315483
>         Memtable Data Size: 14827701
>         Memtable Switch Count: 65
>         Read Count: 5653028
>         Read Latency: 6.888 ms.
>         Write Count: 47666
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 160
>         Compacted row maximum size: 5683855
>         Compacted row mean size: 35031
>
>         Column Family: ColumnFamily16
>         SSTable count: 1
>         Space used (live): 44605875
>         Space used (total): 44605875
>         Memtable Columns Count: 20273
>         Memtable Data Size: 645383
>         Memtable Switch Count: 3
>         Read Count: 373155
>         Read Latency: 0.058 ms.
>         Write Count: 199847
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 209
>         Compacted row maximum size: 82030
>         Compacted row mean size: 617
>
>         Column Family: ColumnFamily17
>         SSTable count: 1
>         Space used (live): 5609
>         Space used (total): 5609
>         Memtable Columns Count: 166
>         Memtable Data Size: 2075
>         Memtable Switch Count: 3
>         Read Count: 17630
>         Read Latency: NaN ms.
>         Write Count: 542
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 522
>         Compacted row maximum size: 563
>         Compacted row mean size: 545
>
>         Column Family: ColumnFamily18
>         SSTable count: 3
>         Space used (live): 20705514
>         Space used (total): 20705514
>         Memtable Columns Count: 164
>         Memtable Data Size: 5892
>         Memtable Switch Count: 3
>         Read Count: 5605
>         Read Latency: NaN ms.
>         Write Count: 318
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 2314
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 521
>         Compacted row maximum size: 1239
>         Compacted row mean size: 604
>
>         Column Family: ColumnFamily19
>         SSTable count: 1
>         Space used (live): 29516821
>         Space used (total): 29516821
>         Memtable Columns Count: 8402
>         Memtable Data Size: 268574
>         Memtable Switch Count: 3
>         Read Count: 7587671
>         Read Latency: 0.193 ms.
>         Write Count: 72597
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 186
>         Compacted row maximum size: 18443
>         Compacted row mean size: 270
>
>         Column Family: ColumnFamily20
>         SSTable count: 1
>         Space used (live): 711618
>         Space used (total): 711618
>         Memtable Columns Count: 16
>         Memtable Data Size: 496
>         Memtable Switch Count: 3
>         Read Count: 454
>         Read Latency: NaN ms.
>         Write Count: 97
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 385
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 238
>         Compacted row maximum size: 426
>         Compacted row mean size: 303
>
>         Column Family: ColumnFamily21
>         SSTable count: 6
>         Space used (live): 499913576
>         Space used (total): 499913576
>         Memtable Columns Count: 79019
>         Memtable Data Size: 4820159
>         Memtable Switch Count: 11
>         Read Count: 65780
>         Read Latency: NaN ms.
>         Write Count: 6129833
>         Write Latency: 0.036 ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 278
>         Compacted row maximum size: 288
>         Compacted row mean size: 284
>
>         Column Family: ColumnFamily22
>         SSTable count: 3
>         Space used (live): 378024
>         Space used (total): 378024
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 724
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>         Column Family: ColumnFamily23
>         SSTable count: 3
>         Space used (live): 11425
>         Space used (total): 11425
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 1
>         Read Count: 578
>         Read Latency: NaN ms.
>         Write Count: 1
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>         Column Family: ColumnFamily24
>         SSTable count: 2
>         Space used (live): 20577518
>         Space used (total): 20577518
>         Memtable Columns Count: 306
>         Memtable Data Size: 12731
>         Memtable Switch Count: 3
>         Read Count: 18
>         Read Latency: NaN ms.
>         Write Count: 3234
>         Write Latency: 0.013 ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 465334
>         Compacted row maximum size: 4552398
>         Compacted row mean size: 2815873
>
>         Column Family: ColumnFamily25
>         SSTable count: 1
>         Space used (live): 2888
>         Space used (total): 2888
>         Memtable Columns Count: 2
>         Memtable Data Size: 68
>         Memtable Switch Count: 3
>         Read Count: 1385944
>         Read Latency: NaN ms.
>         Write Count: 19
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache: disabled
>         Row cache: disabled
>         Compacted row minimum size: 223
>         Compacted row maximum size: 234
>         Compacted row mean size: 230
>
>         Column Family: ColumnFamily26
>         SSTable count: 0
>         Space used (live): 0
>         Space used (total): 0
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 0
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 200000
>         Key cache size: 0
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
> ----------------
> Keyspace: system
>     Read Count: 14
>     Read Latency: 123.1865 ms.
>     Write Count: 6
>     Write Latency: 0.11666666666666667 ms.
>     Pending Tasks: 0
>         Column Family: LocationInfo
>         SSTable count: 1
>         Space used (live): 2871
>         Space used (total): 2871
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 2
>         Read Count: 2
>         Read Latency: NaN ms.
>         Write Count: 6
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 1
>         Key cache size: 1
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 203
>         Compacted row maximum size: 554
>         Compacted row mean size: 378
>
>         Column Family: HintsColumnFamily
>         SSTable count: 2
>         Space used (live): 4876356
>         Space used (total): 4876356
>         Memtable Columns Count: 0
>         Memtable Data Size: 0
>         Memtable Switch Count: 0
>         Read Count: 12
>         Read Latency: NaN ms.
>         Write Count: 0
>         Write Latency: NaN ms.
>         Pending Tasks: 0
>         Key cache capacity: 2
>         Key cache size: 2
>         Key cache hit rate: NaN
>         Row cache: disabled
>         Compacted row minimum size: 0
>         Compacted row maximum size: 0
>         Compacted row mean size: 0
>
>
>
> On 02/28/2012 06:51 PM, Dan Retzlaff wrote:
>
> Hi Stefan. Can you share the output of nodetool cfstats?
>
> On Tue, Feb 28, 2012 at 1:50 AM, Stefan Reek <stefan@unitedgames.com>wrote:
>
>> Hi All,
>>
>> We are running a 3-node cluster with Cassandra 0.6.13.
>> We are in the process of upgrading to 1.x, but can't do so for a while
>> because we can't take the cluster offline.
>> Until now 0.6.13 has run without problems, but lately we are getting some
>> performance issues.
>> We are getting timeouts on both reads and writes.
>> The cluster contains a lot of "old" data which is only read very
>> occasionally and not written to at all.
>> Is it possible for this data to negatively impact the performance of the
>> cluster?
>> If so, would it help to move this data to another columnfamily/keyspace?
>>
>> Cheers,
>>
>> Stefan
>>
>
>
>

--f46d04448035e8198404ba1ecb03
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

First to be clear, I&#39;m not an expert but I suggested &quot;cfstats&quot=
; because it can show unhealthy signs. That said, yours looks okay to me: f=
ew live SSTables per column family, reasonable quantity of data...=A0The ne=
xt things I&#39;d verify are that (1) the JVM isn&#39;t swapping out during=
 these periods of bad performance, (2) GC completes in less than your RPC t=
imeout (check the logs), and (3) your internode network connection isn&#39;=
t degraded by another application.<div>
<br></div><div>The fact that *writes* are timing out is interesting, since =
existing data should have little to no effect on those.</div><div><br></div=
><div>BTW, since ColumnFamily7 sees most of your action, you might consider=
 whether a key/row cache is appropriate. By my count it has 2M rows and 7M =
reads since startup. But this fine-tuning is probably a distraction from wh=
atever&#39;s causing your timeouts.</div>
<div><br></div><div>Hope that helps,</div><div>Dan<br><div><div><div><div><=
div><br><div class=3D"gmail_quote">On Wed, Feb 29, 2012 at 5:14 AM, Stefan =
Reek <span dir=3D"ltr">&lt;<a href=3D"mailto:stefan@unitedgames.com">stefan=
@unitedgames.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><u></u>


 =20

<div bgcolor=3D"#ffffff" text=3D"#000000">
Hi Dan,<br>
<br>
Thanks for answering.<br>
I included the output of cfstats below.<br>
I hope you can say something about our problems with it.<br>
<br>
cheers,<br>
<br>
Stefan<br>
<br>
Keyspace: Keyspace1<br>
=A0=A0=A0 Read Count: 60703419<br>
=A0=A0=A0 Read Latency: 1.1790332096286043 ms.<br>
=A0=A0=A0 Write Count: 105871791<br>
=A0=A0=A0 Write Latency: 0.019847457393065166 ms.<br>
=A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily1<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily2<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 2<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 482828<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 482828<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 9<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 258<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 275<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 266<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily3<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily4<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 19793722<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 19793722<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 306<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 12731<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 2309<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 3234<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: 0.021 ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 465813<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 4784014<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 3295196<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily5<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 6<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 364775507<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 364775507<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 14193<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 552090<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 211451<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 26567<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 187<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 333233<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 662<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily6<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily7<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 12<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 6031951162<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 6031951162<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 480647<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 29424092<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 43<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 7506944<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: 0.761 ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 3022099<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 239<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 93925<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 2737<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily8<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 241104<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 241104<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 2<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 94<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 685<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 143<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 254<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 254<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 479<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 305<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily9<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily10<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 3<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 22874666<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 22874666<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 11886<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 519166<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 73970<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 161<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 103903<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 11504<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily11<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 3<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 6533310<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 6533310<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 8<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 334<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 5431<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 197<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 653<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 262<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 17149<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 359<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily12<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 5504796<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 5504796<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 722<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 5495928<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 6413<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 5784<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 167<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 5498089<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 261974<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily13<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 3<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 18208017<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 18208017<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 40<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 1730<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 4617<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 132<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 1604<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 512<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 1776<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 683<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily14<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 9<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 1631088415<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 1631088415<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 355843<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 15750980<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 1142<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 37874988<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: 0.830 ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 96285506<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: 0.032 ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 599<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 1124<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 861<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily15<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 2<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 69991756<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 69991756<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 315483<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 14827701<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 65<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 5653028<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: 6.888 ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 47666<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 160<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 5683855<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 35031<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily16<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 44605875<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 44605875<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 20273<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 645383<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 373155<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: 0.058 ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 199847<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 209<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 82030<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 617<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily17<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 5609<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 5609<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 166<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 2075<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 17630<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 542<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 522<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 563<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 545<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily18<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 3<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 20705514<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 20705514<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 164<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 5892<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 5605<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 318<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 2314<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 521<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 1239<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 604<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily19<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 29516821<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 29516821<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 8402<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 268574<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 7587671<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: 0.193 ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 72597<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 186<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 18443<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 270<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily20<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 711618<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 711618<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 16<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 496<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 454<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 97<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 385<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 238<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 426<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 303<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily21<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 6<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 499913576<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 499913576<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 79019<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 4820159<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 11<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 65780<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 6129833<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: 0.036 ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 278<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 288<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 284<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily22<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 3<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 378024<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 378024<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 724<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily23<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 3<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 11425<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 11425<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 1<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 578<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 1<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily24<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 2<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 20577518<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 20577518<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 306<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 12731<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 18<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 3234<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: 0.013 ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 465334<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 4552398<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 2815873<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily25<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 2888<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 2888<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 2<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 68<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 3<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 1385944<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 19<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 223<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 234<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 230<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: ColumnFamily26<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 0<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 200000<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<br>
<br>
----------------<br>
Keyspace: system<br>
=A0=A0=A0 Read Count: 14<br>
=A0=A0=A0 Read Latency: 123.1865 ms.<br>
=A0=A0=A0 Write Count: 6<br>
=A0=A0=A0 Write Latency: 0.11666666666666667 ms.<br>
=A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Column Family: LocationInfo<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 1<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 2871<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 2871<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 2<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 2<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 6<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 1<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 1<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 203<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 554<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 378<br>
<br>
=A0=A0=A0 =A0=A0=A0 Column Family: HintsColumnFamily<br>
=A0=A0=A0 =A0=A0=A0 SSTable count: 2<br>
=A0=A0=A0 =A0=A0=A0 Space used (live): 4876356<br>
=A0=A0=A0 =A0=A0=A0 Space used (total): 4876356<br>
=A0=A0=A0 =A0=A0=A0 Memtable Columns Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Data Size: 0<br>
=A0=A0=A0 =A0=A0=A0 Memtable Switch Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Read Count: 12<br>
=A0=A0=A0 =A0=A0=A0 Read Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Write Count: 0<br>
=A0=A0=A0 =A0=A0=A0 Write Latency: NaN ms.<br>
=A0=A0=A0 =A0=A0=A0 Pending Tasks: 0<br>
=A0=A0=A0 =A0=A0=A0 Key cache capacity: 2<br>
=A0=A0=A0 =A0=A0=A0 Key cache size: 2<br>
=A0=A0=A0 =A0=A0=A0 Key cache hit rate: NaN<br>
=A0=A0=A0 =A0=A0=A0 Row cache: disabled<br>
=A0=A0=A0 =A0=A0=A0 Compacted row minimum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row maximum size: 0<br>
=A0=A0=A0 =A0=A0=A0 Compacted row mean size: 0<div><div class=3D"h5"><br>
<br>
<br>
On 02/28/2012 06:51 PM, Dan Retzlaff wrote:
<blockquote type=3D"cite">Hi Stefan.=A0Can you share the output of nodetool=
 cfstats?
  <div><br>
  <div class=3D"gmail_quote">On Tue, Feb 28, 2012 at 1:50 AM, Stefan Reek
  <span dir=3D"ltr">&lt;<a href=3D"mailto:stefan@unitedgames.com" target=3D=
"_blank">stefan@unitedgames.com</a>&gt;</span>
wrote:<br>
  <blockquote class=3D"gmail_quote" style=3D"border-left:1px solid rgb(204,=
204,204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex">Hi
All,<br>
    <br>
We are running a 3-node cluster with Cassandra 0.6.13.<br>
We are in the process of upgrading to 1.x, but can&#39;t do so for a while
because we can&#39;t take the cluster offline.<br>
Until now 0.6.13 has run without problems, but lately we are getting
some performance issues.<br>
We are getting timeouts on both reads and writes.<br>
The cluster contains a lot of &quot;old&quot; data which is only read very
occasionally and not written to at all.<br>
Is it possible for this data to negatively impact the performance of
the cluster?<br>
If so, would it help to move this data to another columnfamily/keyspace?<br=
>
    <br>
Cheers,<br>
    <br>
Stefan<br>
  </blockquote>
  </div>
  <br>
  </div>
</blockquote>
<br>
</div></div></div>

</blockquote></div><br></div></div></div></div></div></div>

--f46d04448035e8198404ba1ecb03--