Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of jbellis@gmail.com designates
 74.125.82.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=uPDO369jeyLrPz3/ThGPbF+trSnTMGq4J51E3R8WSW75a6X6aE+UFOY7c+enMdEhBL
         cKTvPp6GKYNc6anywX0wllTJF23pRX2qNLBN4JVMlUWbA9YU8oveIsET5YRbSR0/9coF
         03DCR9jdsdmcnXE8iQm7MUqH4OW77d2p3VjgA=
MIME-Version: 1.0
In-Reply-To: <AANLkTinaxJ1rmBXH4etBBdNJ_OcYdp5zl86F7g9e84No@mail.gmail.com>
References: <AANLkTinaxJ1rmBXH4etBBdNJ_OcYdp5zl86F7g9e84No@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Mon, 31 May 2010 14:07:16 -0500
Message-ID: <AANLkTikgxDr25ilXixZEs2PHVoTU5UY68TLzgCERM4Sv@mail.gmail.com>
Subject: Re: nodetool cleanup isn't cleaning up?
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

you have replication factor > 1 ?

On Mon, May 31, 2010 at 7:23 AM, Ran Tavory <rantav@gmail.com> wrote:
> I hope I=A0understand=A0nodetool cleanup correctly - it should clean up a=
ll data
> that does not (currently) belong to this node. If so, I think it might no=
t
> be working correctly.
> Look at nodes=A0192.168.252.124 and=A0192.168.252.99 below
> 192.168.252.99Up =A0 =A0 =A0 =A0 279.35 MB =A0 =A0 3544607988759775661076=
818827414252202
> =A0 =A0 =A0|<--|
> 192.168.252.124Up =A0 =A0 =A0 =A0 167.23 MB
> 56713727820156410577229101238628035242 =A0 =A0 | =A0 ^
> 192.168.252.125Up =A0 =A0 =A0 =A0 82.91 MB
> =A085070591730234615865843651857942052863 =A0 =A0 v =A0 |
> 192.168.254.57Up =A0 =A0 =A0 =A0 366.6 MB
> =A0113427455640312821154458202477256070485 =A0 =A0| =A0 ^
> 192.168.254.58Up =A0 =A0 =A0 =A0 88.44 MB
> =A0141784319550391026443072753096570088106 =A0 =A0v =A0 |
> 192.168.254.59Up =A0 =A0 =A0 =A0 88.45 MB
> =A0170141183460469231731687303715884105727 =A0 =A0|-->|
> I wanted 124 to take all the load from 99. So I issued a move command.
> $ nodetool -h cass99 -p 9004 move 56713727820156410577229101238628035243
>
> This command tells 99 to take the space b/w
> (56713727820156410577229101238628035242,=A0567137278201564105772291012386=
28035243]
> which is basically just one item in the token space, almost nothing... I
> wanted it to be very slim (just playing around).
> So, next I get this:
> 192.168.252.124Up =A0 =A0 =A0 =A0 803.33 MB
> 56713727820156410577229101238628035242 =A0 =A0 |<--|
> 192.168.252.99Up =A0 =A0 =A0 =A0 352.85 MB
> 56713727820156410577229101238628035243 =A0 =A0 | =A0 ^
> 192.168.252.125Up =A0 =A0 =A0 =A0 134.24 MB
> 85070591730234615865843651857942052863 =A0 =A0 v =A0 |
> 192.168.254.57Up =A0 =A0 =A0 =A0 676.41 MB
> 113427455640312821154458202477256070485 =A0 =A0| =A0 ^
> 192.168.254.58Up =A0 =A0 =A0 =A0 99.74 MB
> =A0141784319550391026443072753096570088106 =A0 =A0v =A0 |
> 192.168.254.59Up =A0 =A0 =A0 =A0 99.94 MB
> =A0170141183460469231731687303715884105727 =A0 =A0|-->|
> The tokens are correct, but it seems that 99 still has a lot of data. Why=
?
> OK, that might be b/c it didn't delete its moved data.
> So next I issued a nodetool cleanup, which should have taken care of that=
.
> Only that it didn't, the node 99 still has 352 MB of data. Why?
> So, you know what, I waited for 1h. Still no good, data wasn't cleaned up=
.
> I restarted the server. Still, data wasn't cleaned up... I issued a clean=
up
> again... still no good... what's up with this node?
>
>


--=20
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com