Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 43262933D for ; Fri, 16 Mar 2012 18:04:17 +0000 (UTC) Received: (qmail 3860 invoked by uid 500); 16 Mar 2012 18:04:14 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 3835 invoked by uid 500); 16 Mar 2012 18:04:14 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 3823 invoked by uid 99); 16 Mar 2012 18:04:14 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Mar 2012 18:04:14 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [216.82.254.195] (HELO mail200.messagelabs.com) (216.82.254.195) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Mar 2012 18:04:08 +0000 X-Env-Sender: Bryce.Godfrey@azaleos.com X-Msg-Ref: server-4.tower-200.messagelabs.com!1331921025!4684179!1 X-Originating-IP: [96.31.162.253] X-StarScan-Version: 6.5.5; banners=-,-,- X-VirusChecked: Checked Received: (qmail 20092 invoked from network); 16 Mar 2012 18:03:46 -0000 Received: from unknown (HELO smtp.azaleos.net) (96.31.162.253) by server-4.tower-200.messagelabs.com with AES128-SHA encrypted SMTP; 16 Mar 2012 18:03:46 -0000 Received: from FISH-EX2K10-01.azaleos.net ([fe80::389b:5e08:38cd:b989]) by FISH-EX2K10-02.azaleos.net ([fe80::92b:9330:6521:ef18%13]) with mapi id 14.02.0283.003; Fri, 16 Mar 2012 11:03:44 -0700 From: Bryce Godfrey To: "user@cassandra.apache.org" Subject: RE: Large hints column family Thread-Topic: Large hints column family Thread-Index: Ac0CGOOSsB7ZiYpYTpqIVJjUcubtJwAAK5JQACp4g4AAA0n9IAAhfLcAABILDTA= Date: Fri, 16 Mar 2012 18:03:43 +0000 Message-ID: <376CEC01195C894CB9F8A3C274029A96B471D61A@FISH-EX2K10-01.azaleos.net> References: <376CEC01195C894CB9F8A3C274029A96B471C1F7@FISH-EX2K10-01.azaleos.net> <376CEC01195C894CB9F8A3C274029A96B471C20D@FISH-EX2K10-01.azaleos.net> <9C707667-770D-43E7-9F3C-F3EBFB3929CE@thelastpickle.com> <376CEC01195C894CB9F8A3C274029A96B471D05B@FISH-EX2K10-01.azaleos.net> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.200.10.162] Content-Type: multipart/alternative; boundary="_000_376CEC01195C894CB9F8A3C274029A96B471D61AFISHEX2K1001aza_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_376CEC01195C894CB9F8A3C274029A96B471D61AFISHEX2K1001aza_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I took the reset the world approach, things are much better now and the hin= ts table is staying empty. Bit disconcerting that it could get so large an= d not be able to recover itself, but at least there was a solution. Thanks From: aaron morton [mailto:aaron@thelastpickle.com] Sent: Thursday, March 15, 2012 7:24 PM To: user@cassandra.apache.org Subject: Re: Large hints column family These messages make it look like the node is having trouble delivering hint= s. INFO [HintedHandoff:1] 2012-03-13 16:13:34,188 HintedHandOffManager.java (l= ine 284) Endpoint /192.168.20.4 died before hint delivery, aborting INFO [HintedHandoff:1] 2012-03-13 17:03:50,986 HintedHandOffManager.java (l= ine 354) Timed out replaying hints to /192.168.20.3; aborting further deliv= eries Take another look at the logs on this machine and on 20.4 and 20.3. I would be looking int why so many hints are been stored. GC ? are there al= so logs about dropped messages ? If you want to reset the world, make sure the nodes have all run repair and= then drop the hints. Either via JMX or stopped in the node and deleting th= e files on disk. Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 16/03/2012, at 12:58 PM, Bryce Godfrey wrote: We were having some occasional memory pressure issues, but we just added so= me more RAM a few days ago to the nodes and things are running more smoothl= y now, but in general nodes have not been going up and down. I tried to do a "list HintsColumnFamily" from Cassandra-cli and it locks my= Cassandra node and never returns, forcing me to kill the Cassandra process= and restart it to get the node back. Here is my settings which I believe are default since I don't remember chan= ging them: hinted_handoff_enabled: true max_hint_window_in_ms: 3600000 # one hour hinted_handoff_throttle_delay_in_ms: 50 Greping for Hinted in system log I get these INFO [HintedHandoff:1] 2012-03-13 16:13:22,215 HintedHandOffManager.java (l= ine 373) Finished hinted handoff of 852703 rows to endpoint /192.168.20.3 INFO [HintedHandoff:1] 2012-03-13 16:13:34,188 HintedHandOffManager.java (l= ine 284) Endpoint /192.168.20.4 died before hint delivery, aborting INFO [ScheduledTasks:1] 2012-03-13 16:15:32,569 StatusLogger.java (line 65)= HintedHandoff 1 1 0 INFO [HintedHandoff:1] 2012-03-13 16:15:44,362 HintedHandOffManager.java (l= ine 296) Started hinted handoff for token: 11342745564031281485796955865106= 2452224 with IP: /192.168.20.3 INFO [HintedHandoff:1] 2012-03-13 16:21:37,266 HintedHandOffManager.java (l= ine 296) Started hinted handoff for token: 11342745564031281485796955865106= 2452224 with IP: /192.168.20.3 INFO [ScheduledTasks:1] 2012-03-13 16:23:07,662 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-13 16:25:49,330 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-13 16:30:52,503 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-13 16:42:22,202 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [HintedHandoff:1] 2012-03-13 17:03:50,986 HintedHandOffManager.java (l= ine 354) Timed out replaying hints to /192.168.20.3; aborting further deliv= eries INFO [HintedHandoff:1] 2012-03-13 17:03:50,986 ColumnFamilyStore.java (line= 704) Enqueuing flush of Memtable-HintsColumnFamily@661547256(34298224/7446= 5815 serialized/live bytes, 78808 ops) INFO [HintedHandoff:1] 2012-03-13 17:11:00,098 HintedHandOffManager.java (l= ine 373) Finished hinted handoff of 44160 rows to endpoint /192.168.20.3 INFO [HintedHandoff:1] 2012-03-13 17:11:36,596 HintedHandOffManager.java (l= ine 296) Started hinted handoff for token: 56713727820156407428984779325531= 226112 with IP: /192.168.20.4 INFO [ScheduledTasks:1] 2012-03-13 17:12:25,248 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [HintedHandoff:1] 2012-03-13 18:47:56,151 HintedHandOffManager.java (l= ine 296) Started hinted handoff for token: 11342745564031281485796955865106= 2452224 with IP: /192.168.20.3 INFO [ScheduledTasks:1] 2012-03-13 18:50:24,326 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:12:48,177 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:13:57,685 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:14:57,258 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:14:58,260 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:15:59,093 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:16:59,428 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:18:01,862 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:18:01,898 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:19:04,527 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:19:04,541 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:20:07,712 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [ScheduledTasks:1] 2012-03-14 12:20:08,332 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [HintedHandoff:1] 2012-03-14 12:27:13,033 HintedHandOffManager.java (l= ine 296) Started hinted handoff for token: 11342745564031281485796955865106= 2452224 with IP: /192.168.20.3 INFO [ScheduledTasks:1] 2012-03-15 15:05:00,954 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [HintedHandoff:1] 2012-03-15 15:06:07,750 HintedHandOffManager.java (l= ine 354) Timed out replaying hints to /192.168.20.3; aborting further deliv= eries INFO [ScheduledTasks:1] 2012-03-15 15:06:07,802 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [HintedHandoff:1] 2012-03-15 15:06:07,809 ColumnFamilyStore.java (line= 704) Enqueuing flush of Memtable-HintsColumnFamily@254668880(103911/831288= 0 serialized/live bytes, 63877 ops) INFO [ScheduledTasks:1] 2012-03-15 15:07:13,503 StatusLogger.java (line 65)= HintedHandoff 1 2 0 INFO [HintedHandoff:1] 2012-03-15 15:15:43,842 HintedHandOffManager.java (l= ine 296) Started hinted handoff for token: 11342745564031281485796955865106= 2452224 with IP: /192.168.20.3 From: aaron morton [mailto:aaron@thelastpickle.com] Sent: Thursday, March 15, 2012 1:51 AM To: user@cassandra.apache.org Subject: Re: Large hints column family Is there anything going on in the logs ? Are nodes going up and down ? Can = you see any messages about delivering hints ? If the query to read the hints errors it will log "HintsCF getEPPendingHint= s timed out" at INFO level. Also checking, do the hinted_handoff_* settings in cassandra.yaml have the= ir default settings ? Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 15/03/2012, at 8:35 AM, Bryce Godfrey wrote: Forgot to mention that this is on 1.0.8 From: Bryce Godfrey [mailto:Bryce.Godfrey@azaleos.com] Sent: Wednesday, March 14, 2012 12:34 PM To: user@cassandra.apache.org Subject: Large hints column family The system HintsColumnFamily seems large in my cluster, and I want to track= down why that is. I try invoking "listEndpointsPendingHints()" for o.a.c.= db.HintedHandoffManager and it never returns, and also freezes the node tha= t its invoked against. It's a 3 node cluster, and all nodes have been up a= nd running without issue for a while. Any help on where to start with this= ? Column Family: HintsColumnFamily SSTable count: 11 Space used (live): 11271669539 Space used (total): 11271669539 Number of Keys (estimate): 1408 Memtable Columns Count: 338 Memtable Data Size: 0 Memtable Switch Count: 1 Read Count: 3 Read Latency: 4354.669 ms. Write Count: 848 Write Latency: 0.029 ms. Pending Tasks: 0 Bloom Filter False Postives: 0 Bloom Filter False Ratio: 0.00000 Bloom Filter Space Used: 12656 Key cache capacity: 14 Key cache size: 11 Key cache hit rate: 0.6666666666666666 Row cache: disabled Compacted row minimum size: 105779 Compacted row maximum size: 7152383774 Compacted row mean size: 590818614 Thanks, Bryce --_000_376CEC01195C894CB9F8A3C274029A96B471D61AFISHEX2K1001aza_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I took the reset the worl= d approach, things are much better now and the hints table is staying empty= .  Bit disconcerting that it could get so large and not be able to recover itself, but at least there was a solution.  Thanks=

 <= /p>

 <= /p>

From: aaron mo= rton [mailto:aaron@thelastpickle.com]
Sent: Thursday, March 15, 2012 7:24 PM
To: user@cassandra.apache.org
Subject: Re: Large hints column family

 

These messages make it look like the node is having = trouble delivering hints. 

INFO [HintedHandoff:1] 20= 12-03-13 16:13:34,188 HintedHandOffManager.java (line 284) Endpoint /192.16= 8.20.4 died before hint delivery, aborting

INFO [HintedHandoff:1] 20= 12-03-13 17:03:50,986 HintedHandOffManager.java (line 354) Timed out replay= ing hints to /192.168.20.3; aborting further deliveries

 

Take another look at the = logs on this machine and on 20.4 and 20.3. 

 

I would be looking int why so many hints are been stored. GC ? are = there also logs about dropped messages ? 

 

If you want to reset the world, make sure the nodes have all run re= pair and then drop the hints. Either via JMX or stopped in the node and deleting the files on disk. 

 

Cheers

 

-----------------

Aaron Morton

Freelance Developer<= /o:p>

@aaronmorton

 

On 16/03/2012, at 12:58 PM, Bryce Godfrey wrote:



We were having some occas= ional memory pressure issues, but we just added some more RAM a few days ag= o to the nodes and things are running more smoothly now, but in general nodes have not been going up and down.

 <= /p>

I tried to do a “li= st HintsColumnFamily” from Cassandra-cli and it locks my Cassandra no= de and never returns, forcing me to kill the Cassandra process and restart it to get the node back.

 <= /p>

Here is my settings which= I believe are default since I don’t remember changing them:

 <= /p>

hinted_handoff_enabled: t= rue

max_hint_window_in_ms: 36= 00000 # one hour

hinted_handoff_throttle_d= elay_in_ms: 50

 <= /p>

Greping for Hinted in sys= tem log I get these

INFO [HintedHandoff:1] 20= 12-03-13 16:13:22,215 HintedHandOffManager.java (line 373) Finished hinted = handoff of 852703 rows to endpoint /192.168.20.3

INFO [HintedHandoff:1] 20= 12-03-13 16:13:34,188 HintedHandOffManager.java (line 284) Endpoint /192.16= 8.20.4 died before hint delivery, aborting

INFO [ScheduledTasks:1] 2= 012-03-13 16:15:32,569 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  1         0

INFO [HintedHandoff:1] 20= 12-03-13 16:15:44,362 HintedHandOffManager.java (line 296) Started hinted h= andoff for token: 113427455640312814857969558651062452224 with IP: /192.168.20.3

INFO [HintedHandoff:1] 20= 12-03-13 16:21:37,266 HintedHandOffManager.java (line 296) Started hinted h= andoff for token: 113427455640312814857969558651062452224 with IP: /192.168.20.3

INFO [ScheduledTasks:1] 2= 012-03-13 16:23:07,662 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-13 16:25:49,330 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-13 16:30:52,503 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1       &n= bsp; 2         0

INFO [ScheduledTasks:1] 2= 012-03-13 16:42:22,202 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [HintedHandoff:1] 20= 12-03-13 17:03:50,986 HintedHandOffManager.java (line 354) Timed out replay= ing hints to /192.168.20.3; aborting further deliveries

INFO [HintedHandoff:1] 20= 12-03-13 17:03:50,986 ColumnFamilyStore.java (line 704) Enqueuing flush of = Memtable-HintsColumnFamily@661547256(34298224/74465815 serialized/live bytes, 78808 ops)

INFO [HintedHandoff:1] 20= 12-03-13 17:11:00,098 HintedHandOffManager.java (line 373) Finished hinted = handoff of 44160 rows to endpoint /192.168.20.3

INFO [HintedHandoff:1] 20= 12-03-13 17:11:36,596 HintedHandOffManager.java (line 296) Started hinted h= andoff for token: 56713727820156407428984779325531226112 with IP: /192.168.20.4

INFO [ScheduledTasks:1] 2= 012-03-13 17:12:25,248 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [HintedHandoff:1] 20= 12-03-13 18:47:56,151 HintedHandOffManager.java (line 296) Started hinted h= andoff for token: 113427455640312814857969558651062452224 with IP: /192.168.20.3

INFO [ScheduledTasks:1] 2= 012-03-13 18:50:24,326 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:12:48,177 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:13:57,685 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1       &n= bsp; 2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:14:57,258 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:14:58,260 StatusLogger.java (line 65) HintedHandoff  = ;             &= nbsp;     1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:15:59,093 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:16:59,428 StatusLogger.java (line 65) HintedHandoff  = ;             &= nbsp;     1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:18:01,862 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:18:01,898 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:19:04,527 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:19:04,541 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:20:07,712 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [ScheduledTasks:1] 2= 012-03-14 12:20:08,332 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [HintedHandoff:1] 20= 12-03-14 12:27:13,033 HintedHandOffManager.java (line 296) Started hinted h= andoff for token: 113427455640312814857969558651062452224 with IP: /192.168.20.3

INFO [ScheduledTasks:1] 2= 012-03-15 15:05:00,954 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [HintedHandoff:1] 20= 12-03-15 15:06:07,750 HintedHandOffManager.java (line 354) Timed out replay= ing hints to /192.168.20.3; aborting further deliveries

INFO [ScheduledTasks:1] 2= 012-03-15 15:06:07,802 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [HintedHandoff:1] 20= 12-03-15 15:06:07,809 ColumnFamilyStore.java (line 704) Enqueuing flush of = Memtable-HintsColumnFamily@254668880(103911/8312880 serialized/live bytes, 63877 ops)

INFO [ScheduledTasks:1] 2= 012-03-15 15:07:13,503 StatusLogger.java (line 65) HintedHandoff  = ;            &n= bsp;      1      &nb= sp;  2         0

INFO [HintedHandoff:1] 20= 12-03-15 15:15:43,842 HintedHandOffManager.java (line 296) Started hinted h= andoff for token: 113427455640312814857969558651062452224 with IP: /192.168.20.3

 <= /p>

 <= /p>

From: aaron morton [mailto:aaron@thelast= pickle.com]&nb= sp;
Sent: Thursday, Ma= rch 15, 2012 1:51 AM
To: user@cassandra.apache.org=
Subject: Re: Large= hints column family

 

Is there anything going on in the logs ? Are nodes g= oing up and down ? Can you see any messages about delivering hints ? <= o:p>

 

If the query to read the hints errors it will log &q= uot;HintsCF getEPPendingHints timed out" at INFO level. 

 

Also checking, do the hinted_handoff_*  se= ttings in cassandra.yaml have their default settings ?

 

Cheers

 

-----------------<= o:p>

Aaron Morton<= /o:p>

Freelance Developer

@aaronmorton<= /o:p>

 

On 15/03/2012, at 8:35 AM, Bryce Godfrey wrote:=




Forgot to mention that th= is is on 1.0.8

 <= /p>

From: Bryce Godfrey [mailto:Bryce.Go= dfrey@azaleos.com] 
Sent: Wednesday, M= arch 14, 2012 12:34 PM
To: user@cassandra.apache.org=
Subject: Large hin= ts column family

 

The system HintsColumnFamily seems larg= e in my cluster, and I want to track down why that is.  I try invoking= “listEndpointsPendingHints()” for o.a.c.db.HintedHandoffManage= r and it never returns, and also freezes the node that its invoked against.&= nbsp; It’s a 3 node cluster, and all nodes have been up and running w= ithout issue for a while.  Any help on where to start with this?

 

      &nb= sp;        Column Family: HintsColumnFam= ily

      &nb= sp;         SSTable count: 11

      &nb= sp;         Space used (live): 1127= 1669539

      &nb= sp;         Space used (total): 112= 71669539

      &nb= sp;         Number of Keys (estimat= e): 1408

      &nb= sp;         Memtable Columns Count:= 338

      &nb= sp;         Memtable Data Size: 0

      &nb= sp;         Memtable Switch Count: = 1

      &nb= sp;         Read Count: 3

      &nb= sp;         Read Latency: 4354.669 = ms.

      &nb= sp;         Write Count: 848=

      &nb= sp;         Write Latency: 0.029 ms= .

      &nb= sp;         Pending Tasks: 0=

      &nb= sp;         Bloom Filter False Post= ives: 0

      &nb= sp;         Bloom Filter False Rati= o: 0.00000

      &nb= sp;         Bloom Filter Space Used= : 12656

      &nb= sp;         Key cache capacity: 14<= /span>

      &nb= sp;         Key cache size: 11

      &nb= sp;         Key cache hit rate: 0.6= 666666666666666

      &nb= sp;         Row cache: disabled

      &nb= sp;         Compacted row minimum s= ize: 105779

      &nb= sp;         Compacted row maximum s= ize: 7152383774

      &nb= sp;         Compacted row mean size= : 590818614

 

Thanks,

Bryce

 

--_000_376CEC01195C894CB9F8A3C274029A96B471D61AFISHEX2K1001aza_--