From user-return-36315-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Tue Sep 3 16:13:03 2013 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 151FB105A4 for ; Tue, 3 Sep 2013 16:13:03 +0000 (UTC) Received: (qmail 5319 invoked by uid 500); 3 Sep 2013 16:12:56 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 4644 invoked by uid 500); 3 Sep 2013 16:12:52 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 4156 invoked by uid 99); 3 Sep 2013 16:12:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Sep 2013 16:12:52 +0000 X-ASF-Spam-Status: No, hits=2.6 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,TRACKER_ID,UNPARSEABLE_RELAY X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [216.82.254.107] (HELO mail1.bemta7.messagelabs.com) (216.82.254.107) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Sep 2013 16:12:37 +0000 Received: from [216.82.254.195:41635] by server-11.bemta-7.messagelabs.com id A2/15-17828-F5A06225; Tue, 03 Sep 2013 16:12:15 +0000 X-Env-Sender: Chris.Lohfink@digi.com X-Msg-Ref: server-12.tower-200.messagelabs.com!1378224734!19820507!1 X-Originating-IP: [66.77.174.14] X-StarScan-Received: X-StarScan-Version: 6.9.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 1865 invoked from network); 3 Sep 2013 16:12:15 -0000 Received: from mail.mx4.digi.com (HELO mcl-sms-ns2.DIGI.COM) (66.77.174.14) by server-12.tower-200.messagelabs.com with RC4-SHA encrypted SMTP; 3 Sep 2013 16:12:15 -0000 Received: from mcl-sms-exch01.digi.com (10.5.8.50) by mail.mx2.digi.com (172.16.1.14) with Microsoft SMTP Server (TLS) id 8.3.327.1; Tue, 3 Sep 2013 11:12:27 -0500 Received: from mtk-sms-exch01.digi.com ([10.10.8.100]) by mcl-sms-exch01.digi.com ([10.5.8.50]) with mapi; Tue, 3 Sep 2013 11:12:14 -0500 From: "Lohfink, Chris" To: "user@cassandra.apache.org" Date: Tue, 3 Sep 2013 11:12:19 -0500 Subject: RE: read ? Thread-Topic: read ? Thread-Index: AQHOqKhF/UGE2JTS20azxDGk9lbrxZm0LhHQ Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_D7797C4132D4A7458DA14B70BB00708AE173E4224Dmtksmsexch01d_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_D7797C4132D4A7458DA14B70BB00708AE173E4224Dmtksmsexch01d_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable To get an accurate picture you should force a full GC on each node, the hea= p utilization can be misleading since there can be a lot of things in the h= eap with no strong references. There is a number of factors that can lead to this. For a true comparison = I would recommend using jconsole and call dumpHeap on com.sun.management:ty= pe=3DHotSpotDiagnostic with the 2nd param true (force GC). Then open the h= eap dump up in a tool like yourkit and you will get a better comparison and= also it will tell you what it is that's taking the space. Chris From: Langston, Jim [mailto:Jim.Langston@compuware.com] Sent: Tuesday, September 03, 2013 8:20 AM To: user@cassandra.apache.org Subject: read ? Hi all, Quick question I currently am looking at a 4 node cluster and I have currently stopped all= writing to Cassandra, with the reads continuing. I'm trying to understand the utiliza= tion of memory within the JVM. nodetool info on each of the nodes shows them all growing in footprint, 2 of the three at a greater rate. On the restart of C= assandra each were at about 100MB, after 2 days, each of the following are at: Heap Memory (MB) : 798.41 / 3052.00 Heap Memory (MB) : 370.44 / 3052.00 Heap Memory (MB) : 549.73 / 3052.00 Heap Memory (MB) : 481.89 / 3052.00 Ring configuration: Address Rack Status State Load Owns = Token = 127605887595351923798765477786913079296 x 1d Up Normal 4.38 GB 25.00% 0 x 1d Up Normal 4.17 GB 25.00% 42535295= 865117307932921825928971026432 x 1d Up Normal 4.19 GB 25.00% 85070591= 730234615865843651857942052864 x 1d Up Normal 4.14 GB 25.00% 12760588= 7595351923798765477786913079296 What I'm not sure of is what the growth is different between each ? and why that growth is being created by activity that is read only. Is Cassandra caching and holding the read data ? I currently have caching turned off for the key/row. Also as part of the in= fo command Key Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, = NaN recent hit rate, 14400 save period in seconds Row Cache : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests, = NaN recent hit rate, 0 save period in seconds Thanks, Jim --_000_D7797C4132D4A7458DA14B70BB00708AE173E4224Dmtksmsexch01d_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

To get an= accurate picture you should force a full GC on each node, the heap utiliza= tion can be misleading since there can be a lot of things in the heap with = no strong references.

 

There is a number of f= actors that can lead to this.  For a true comparison I would recommend= using jconsole and call dumpHeap on com.sun.management:type=3DHotSpotDiagn= ostic with the 2nd param true (force GC).  Then open the he= ap dump up in a tool like yourkit and you will get a better comparison and = also it will tell you what it is that’s taking the space.<= /span>

 

Chris

=  

From:<= span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Langston= , Jim [mailto:Jim.Langston@compuware.com]
Sent: Tuesday, Septemb= er 03, 2013 8:20 AM
To: user@cassandra.apache.org
Subject:<= /b> read ?

 = ;

Hi all,

 

Quick question

 

I currently am looking at a 4 node cluster and I have currently = stopped all writing to 

Cassandra,  with the reads continuing. I'm trying to under= stand the utilization

= of memory within the JVM. nodetool info on each of the nodes shows the= m all 

growi= ng in footprint, 2 of the three at a greater rate. On the restart of Cassan= dra

each were at = about 100MB, after 2 days, each of the following are at:<= /p>

 

<= div>

Heap Memory (MB) : 798.41 / 3052.00

 

Heap Memory (MB) : 370.44 / 3052.00<= o:p>

 =

Heap Memory (MB) : 549.73 /= 3052.00

&= nbsp;

Heap Memory (MB)= : 481.89 / 3052.00

 

Ring c= onfiguration:

 

Addres= s         Rack        Status State =   Load            Owns     &nb= sp;          Token

             =                     &nbs= p;                     &n= bsp;                     =  127605887595351923798765477786913079296

x     1d        =  Up     Normal  4.38 GB         25= .00%              0

x   1d       &nbs= p;  Up     Normal  4.17 GB         = 25.00%              42535295865117307932= 921825928971026432

x   1d       &nbs= p;  Up     Normal  4.19 GB         = 25.00%              85070591730234615865= 843651857942052864

x   1d       &nbs= p;  Up     Normal  4.14 GB         = 25.00%              12760588759535192379= 8765477786913079296

 

<= o:p> 

What I'm no= t sure of is what the growth is different between each ? and why=

that growth is being create= d by activity that is read only.

 

Is Cassandra caching and holding the read data ? 

 

<= /div>

I currently have caching turned off for= the key/row. Also as part of the info command

<= div>

 

<= p class=3DMsoNormal>Key Cache        : size 0 (byt= es), capacity 0 (bytes), 0 hits, 0 requests, NaN recent hit rate, 14400 sav= e period in seconds

Row Cache        : size 0 (bytes), capacity 0 (bytes= ), 0 hits, 0 requests, NaN recent hit rate, 0 save period in seconds

 =

 

 

Thanks,

 

Jim

= --_000_D7797C4132D4A7458DA14B70BB00708AE173E4224Dmtksmsexch01d_--