Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D98AB95F3 for ; Thu, 19 Jan 2012 14:19:36 +0000 (UTC) Received: (qmail 66185 invoked by uid 500); 19 Jan 2012 14:19:31 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 65375 invoked by uid 500); 19 Jan 2012 14:19:31 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 65172 invoked by uid 99); 19 Jan 2012 14:19:30 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jan 2012 14:19:30 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL,TO_NO_BRKTS_PCNT,TRACKER_ID X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jan 2012 14:19:23 +0000 Received: by vbbfr13 with SMTP id fr13so1495601vbb.31 for ; Thu, 19 Jan 2012 06:19:02 -0800 (PST) MIME-Version: 1.0 Received: by 10.52.94.148 with SMTP id dc20mr12712008vdb.109.1326982741083; Thu, 19 Jan 2012 06:19:01 -0800 (PST) Received: by 10.52.170.14 with HTTP; Thu, 19 Jan 2012 06:19:01 -0800 (PST) In-Reply-To: <3760B7EC-9366-445E-9EEE-541F1DD991C1@thelastpickle.com> References: <004a01ccd436$41307900$c3916b00$@com> <4F15FE7C.1040001@morningstar.com> <6F3F8891-48AC-4A67-81E5-451A64C9A375@thelastpickle.com> <009401ccd5c7$c7f4f730$57dee590$@com> <3760B7EC-9366-445E-9EEE-541F1DD991C1@thelastpickle.com> Date: Thu, 19 Jan 2012 15:19:01 +0100 Message-ID: Subject: Re: nodetool ring question From: "R. Verlangen" To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=bcaec5015f2be7c80c04b6e23eb4 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5015f2be7c80c04b6e23eb4 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I will have a look very soon and if I find something I'll let you know. Thank you in advance! 2012/1/19 aaron morton > Michael, Robin > > Let us know if the reported live load is increasing and diverging from th= e > on disk size. > > If it is can you check nodetool cfstats and find an example of a > particular CF where Space Used Live has diverged from the on disk size. T= he > provide the schema for the CF and any other info that may be handy. > > Cheers > > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 18/01/2012, at 10:58 PM, Michael Vaknine wrote: > > I did restart the cluster and now it is normal 5GB.**** > ** ** > *From:* R. Verlangen [mailto:robin@us2.nl] > *Sent:* Wednesday, January 18, 2012 11:32 AM > *To:* user@cassandra.apache.org > *Subject:* Re: nodetool ring question**** > ** ** > > I also have this problem. My data on nodes grows to roughly 30GB. After a > restart only 5GB remains. Is a factor 6 common for Cassandra?**** > 2012/1/18 aaron morton **** > Good idea Jeremiah, are you using compression Michael ? **** > ** ** > Scanning through the CF stats this jumps out=85**** > ** ** > Column Family: Attractions**** > SSTable count: 3**** > Space used (live): 27542876685**** > Space used (total): 1213220387**** > Thats 25Gb of live data but only 1.3GB total. **** > ** ** > Otherwise want to see if a restart fixes it :) Would be interesting to > know if it's wrong from the start or drifts during streaming or compactio= n. > **** > ** ** > Cheers**** > ** ** > -----------------**** > Aaron Morton**** > Freelance Developer**** > @aaronmorton**** > http://www.thelastpickle.com**** > ** ** > On 18/01/2012, at 12:04 PM, Jeremiah Jordan wrote:**** > > > **** > There were some nodetool ring load reporting issues with early version of > 1.0.X don't remember when they were fixed, but that could be your issue. > Are you using compressed column families, a lot of the issues were with > those. > Might update to 1.0.7. > > -Jeremiah > > On 01/16/2012 04:04 AM, Michael Vaknine wrote:**** > Hi,**** > **** > I have a 4 nodes cluster 1.0.3 version**** > **** > This is what I get when I run nodetool ring**** > **** > Address DC Rack Status State Load > Owns Token**** > > 127605887595351923798765477786913079296**** > 10.8.193.87 datacenter1 rack1 Up Normal 46.47 GB > 25.00% 0**** > 10.5.7.76 datacenter1 rack1 Up Normal 48.01 GB > 25.00% 42535295865117307932921825928971026432**** > 10.8.189.197 datacenter1 rack1 Up Normal 53.7 GB > 25.00% 85070591730234615865843651857942052864**** > 10.5.3.17 datacenter1 rack1 Up Normal 43.49 GB > 25.00% 127605887595351923798765477786913079296**** > **** > I have finished running repair on all 4 nodes.**** > **** > I have less then 10 GB on the /var/lib/cassandra/data/ folders**** > **** > My question is Why nodetool reports almost 50 GB on each node?**** > **** > Thanks**** > Michael**** > ** ** > > > --bcaec5015f2be7c80c04b6e23eb4 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I will have a look very soon and if I find something I'll let you know.=

Thank you in advance!

2012/1/19 aaron morton <aaron@thelastpickle.com>
Michael,= Robin

Let = us know if the reported live load is increasing and diverging from the on d= isk size.

If it is c= an you check nodetool cfstats and find an example of a particular CF where = Space Used Live has diverged from the on disk size. The provide the schema = for the CF and any other info that may be handy.=A0

Cheers


<= div> <= div style=3D"word-wrap:break-word">
-----------------
Aaron Morton
Freelance Deve= loper
@aaronmorton

On 18/01/2012, at 10:58 PM, Mich= ael Vaknine wrote:

= I did restart the cluster and now it is normal 5GB.
=A0
From:= =A0R. Verlangen [mailto:ro= bin@us2.nl]=A0
Sent:=A0Wednesday, January 18, 2012 11:32 AM
To:<= /b>=A0user@cassandra.apache.org
Subject:=A0Re: = nodetool ring question
=A0

I also have this problem. My data on nodes grows to roughly 30GB. After a r= estart only 5GB remains. Is a factor 6 common for Cassandra?<= /p>

2012/1/18 aaron morton <aaron@thelastpi= ckle.com>
Good idea Jeremiah, are you using compression Michael ?=A0
=A0
Scanning through the CF stats this jumps out= =85
=A0
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Column Family: Attractions
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 SSTable count: 3
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Space used (live): 27= 542876685
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Space used (total): 1= 213220387
Thats 25Gb of live data but only 1.3GB total.=A0
=A0
Otherwise want to see if a restart f= ixes it :) Would be interesting to know if it's wrong from the start or= drifts during streaming or compaction.=A0
=A0
Cheers
=A0
-----------------
Aar= on Morton
Fre= elance Developer
@aa= ronmorton
=A0
= On 18/01/2012, at 12:04 PM, Jeremiah Jordan wrote:

<= br>
There were some nodetool ring load reporting issues with early version of 1= .0.X don't remember when they were fixed, but that could be your issue.= =A0 Are you using compressed column families, a lot of the issues were with= those.
Might update to 1.0.7.

-Jeremiah

On 01/16/2012 04:04 AM, Mich= ael Vaknine wrote:
Hi,
=A0
I have a 4 nodes cluster 1.0.3 version
=A0
This = is what I get when I run nodetool ring
=A0
Address=A0=A0=A0=A0=A0=A0=A0=A0 DC=A0=A0=A0=A0=A0=A0=A0= =A0=A0 Rack=A0=A0=A0=A0=A0=A0=A0 Status State=A0=A0 Load=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 Owns=A0=A0=A0 Token
=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 127605887595351923798765477786913079296
10.8.= 193.87=A0=A0=A0=A0 datacenter1 rack1=A0=A0=A0=A0=A0=A0 Up=A0=A0=A0=A0 Norma= l=A0 46.47 GB=A0=A0=A0=A0=A0=A0=A0 25.00%=A0 0
10.5.= 7.76=A0=A0=A0=A0=A0=A0 datacenter1 rack1=A0=A0=A0=A0=A0=A0 Up=A0=A0=A0=A0 N= ormal=A0 48.01 GB=A0=A0=A0=A0=A0=A0=A0 25.00%=A0 42535295865117307932921825= 928971026432
10.8.= 189.197=A0=A0=A0 datacenter1 rack1=A0=A0=A0=A0=A0=A0 Up=A0=A0=A0=A0 Normal= =A0 53.7 GB=A0=A0=A0=A0=A0=A0=A0=A0 25.00%=A0 85070591730234615865843651857= 942052864
10.5.= 3.17=A0=A0=A0=A0=A0=A0 datacenter1 rack1=A0=A0=A0=A0=A0=A0 Up=A0=A0=A0=A0 N= ormal=A0 43.49 GB=A0=A0=A0=A0=A0=A0=A0 25.00%=A0 12760588759535192379876547= 7786913079296
=A0
I have finished running repair on all 4 nodes.
=A0<= /u>
I hav= e less then 10 GB on the /var/lib/cassandra/data/ folders
=A0
My question is Why nodetool reports almost 50 GB on each node?
Thanks
Michael
=A0


--bcaec5015f2be7c80c04b6e23eb4--