Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1DDA417E91 for ; Mon, 27 Oct 2014 13:57:26 +0000 (UTC) Received: (qmail 80147 invoked by uid 500); 27 Oct 2014 13:57:23 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 80113 invoked by uid 500); 27 Oct 2014 13:57:23 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 80103 invoked by uid 99); 27 Oct 2014 13:57:23 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Oct 2014 13:57:23 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of bluethundr@gmail.com designates 74.125.82.49 as permitted sender) Received: from [74.125.82.49] (HELO mail-wg0-f49.google.com) (74.125.82.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Oct 2014 13:57:19 +0000 Received: by mail-wg0-f49.google.com with SMTP id x13so2007779wgg.20 for ; Mon, 27 Oct 2014 06:55:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=RuTBCLgi/YPxgO+224euphwoAdnMEG2Tf5oMG5FIaXQ=; b=qBTMBhQre1r9uJA3OZ9CkC48ekgcFaHZK4eF+78rTbeDSEj+zowIxNXWH/IJ5fEKUH nnbjOb4nxm9tkXIjStrZuBgbecnhtV9FxsBB8SkGgWTp2GOEIY8d5+RJxjxCo+WGttDH p8FI5uV/+NqxJg6WP6qKvPSTUI3e3VBJ3FDVdq0M0sxRYjR1GFYAgkSng5+zMFwDRipK EDHnNngf+9G273Gi+d83P0oRUpBChWJ0VeHEvYWOeHfexJW5uAuFE76fyaY9pAnbb1Z3 Ig1JPkQUHqrFohW/O7/nmBLS31ENpr+q7LV3881pmAjusByRzoYlBICDKunDm4y1Dxnc uXeQ== MIME-Version: 1.0 X-Received: by 10.180.100.136 with SMTP id ey8mr21075665wib.83.1414418127920; Mon, 27 Oct 2014 06:55:27 -0700 (PDT) Received: by 10.216.90.134 with HTTP; Mon, 27 Oct 2014 06:55:27 -0700 (PDT) In-Reply-To: References: <1914948500.735312.1414403180734.JavaMail.yahoo@jws10654.mail.bf1.yahoo.com> Date: Mon, 27 Oct 2014 09:55:27 -0400 Message-ID: Subject: Re: decommissioning a cassandra node From: Tim Dunphy To: "user@cassandra.apache.org" Content-Type: multipart/alternative; boundary=f46d0444ec9914247e050667e299 X-Virus-Checked: Checked by ClamAV on apache.org --f46d0444ec9914247e050667e299 Content-Type: text/plain; charset=UTF-8 > > "Also, is there any document that explains what all the nodetool > abbreviations (UN, UL) stand for?" > --> The documentation is in the command output itself > Datacenter: datacenter1 > ======================= > > *Status=Up/Down* > *|/ State=Normal/Leaving/Joining/Moving*-- Address Load > Tokens Owns Host ID Rack > UN 162.243.86.41 1.08 MB 1 0.1% > e945f3b5-2e3e-4a20-b1bd-e30c474a7634 rack1 > UL 162.243.109.94 1.28 MB 256 99.9% > fd2f76ae-8dcf-4e93-a37f-bf1e9088696e rack1 > U = Up, D = Down > N = Normal, L = Leaving, J = Joining and M = Moving Ok, got it, thanks! Can someone suggest a good way to fix a node that is in an UL state? Thanks Tim On Mon, Oct 27, 2014 at 9:46 AM, DuyHai Doan wrote: > "Also, is there any document that explains what all the nodetool > abbreviations (UN, UL) stand for?" > > --> The documentation is in the command output itself > > Datacenter: datacenter1 > ======================= > *Status=Up/Down* > *|/ State=Normal/Leaving/Joining/Moving* > -- Address Load Tokens Owns Host ID > Rack > UN 162.243.86.41 1.08 MB 1 0.1% > e945f3b5-2e3e-4a20-b1bd-e30c474a7634 rack1 > UL 162.243.109.94 1.28 MB 256 99.9% > fd2f76ae-8dcf-4e93-a37f-bf1e9088696e rack1 > > U = Up, D = Down > N = Normal, L = Leaving, J = Joining and M = Moving > > On Mon, Oct 27, 2014 at 2:42 PM, Tim Dunphy wrote: > >> As I see the state 162.243.109.94 is UL(Up/Leaving) so maybe this is >>> causing the problem >> >> >> OK, that's an interesting observation.How do you fix a node that is an UL >> state? What causes this? >> >> Also, is there any document that explains what all the nodetool >> abbreviations (UN, UL) stand for? >> >> On Mon, Oct 27, 2014 at 5:46 AM, jivko donev wrote: >> >>> As I see the state 162.243.109.94 is UL(Up/Leaving) so maybe this is >>> causing the problem. >>> >>> >>> On Sunday, October 26, 2014 11:57 PM, Tim Dunphy >>> wrote: >>> >>> >>> Hey all, >>> >>> I'm trying to decommission a node. >>> >>> First I'm getting a status: >>> >>> [root@beta-new:/usr/local] #nodetool status >>> Note: Ownership information does not include topology; for complete >>> information, specify a keyspace >>> Datacenter: datacenter1 >>> ======================= >>> Status=Up/Down >>> |/ State=Normal/Leaving/Joining/Moving >>> -- Address Load Tokens Owns Host ID >>> Rack >>> UN 162.243.86.41 1.08 MB 1 0.1% >>> e945f3b5-2e3e-4a20-b1bd-e30c474a7634 rack1 >>> UL 162.243.109.94 1.28 MB 256 99.9% >>> fd2f76ae-8dcf-4e93-a37f-bf1e9088696e rack1 >>> >>> >>> But when I try to decommission the node I get this message: >>> >>> [root@beta-new:/usr/local] #nodetool -h 162.243.86.41 decommission >>> nodetool: Failed to connect to '162.243.86.41:7199' - >>> NoSuchObjectException: 'no such object in table'. >>> >>> Yet I can telnet to that host on that port just fine: >>> >>> [root@beta-new:/usr/local] #telnet 162.243.86.41 7199 >>> Trying 162.243.86.41... >>> Connected to 162.243.86.41. >>> Escape character is '^]'. >>> >>> >>> And I have verified that cassandra is running and accessible via cqlsh >>> on the other machine. >>> >>> What could be going wrong? >>> >>> Thanks >>> Tim >>> >>> >>> -- >>> GPG me!! >>> >>> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B >>> >>> >>> >>> >> >> >> -- >> GPG me!! >> >> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B >> >> > -- GPG me!! gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B --f46d0444ec9914247e050667e299 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
"Also, is there any document that explains what all the nodetool abb= reviations (UN, UL) stand for?"=C2=A0
--> The docume= ntation is in the command output itself
Datacenter: datacenter1=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
= Status=3DUp/Down
|/ State=3DNormal/Leaving/Joining/Moving
<= /b>-- =C2=A0Address =C2=A0 =C2=A0 =C2=A0 =C2=A0 Load =C2=A0 =C2=A0 =C2=A0 T= okens =C2=A0Owns =C2=A0 =C2=A0Host ID =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Rack<= br>UN =C2=A0162.243.86.41 =C2=A0 1.08 MB =C2=A0 =C2=A01 =C2=A0 =C2=A0 =C2= =A0 0.1% =C2=A0 =C2=A0e945f3b5-2e3e-4a20-b1bd-e30c474a7634 =C2=A0rack1
U= L =C2=A0162.243.109.94 =C2=A01.28 MB =C2=A0 =C2=A0256 =C2=A0 =C2=A0 99.9% = =C2=A0 fd2f76ae-8dcf-4e93-a37f-bf1e9088696e =C2=A0rack1
U =3D Up, D =3D = Down
N =3D Normal, L =3D Leaving, J =3D Joining and M =3D Moving

Ok, got it, thanks!=C2=A0

Can someone suggest a good way to fix a node that is in an UL state?
=

Thanks
Tim=C2=A0

On Mon, Oct 27, 2014 at 9:46 A= M, DuyHai Doan <doanduyhai@gmail.com> wrote:
"Also, is there any documen= t that explains what all the nodetool abbreviations (UN, UL) stand for?&quo= t;=C2=A0

--> The documentation is in the = command output itself

Datacenter: datacenter= 1
=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Status=3DUp/Down
|/ State= =3DNormal/Leaving/Joining/Moving
-- =C2=A0Address =C2=A0 =C2=A0 =C2=A0 =C2=A0 Load= =C2=A0 =C2=A0 =C2=A0 Tokens =C2=A0Owns =C2=A0 =C2=A0Host ID =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 Rack
UN =C2=A0162.243.86.41 =C2=A0 1.08 MB =C2=A0 =C2=A01 =C2=A0 =C2= =A0 =C2=A0 0.1% =C2=A0 =C2=A0e945f3b5-2e3e-4a20-b1bd-e30c474a7634 =C2=A0rac= k1
UL =C2= =A0162.243.109.94 =C2=A01.28 MB =C2=A0 =C2=A0256 =C2=A0 =C2=A0 99.9% =C2=A0= fd2f76ae-8dcf-4e93-a37f-bf1e9088696e =C2=A0rack1
<= div style=3D"font-family:arial,sans-serif;font-size:13px">
U =3D Up, D =3D Down
N =3D Normal,= L =3D Leaving, J =3D Joining and M =3D Moving

On Mon, Oct 27, 2014 at 2:42 PM, Tim Dunphy <bluethundr@gmail.com= > wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;pa= dding-left:1ex">As I see the state=C2=A0162.243.1= 09.94 is UL(Up/Leaving) so maybe this is causing the problem

OK, that's an interesting observation.How do y= ou fix a node that is an UL state? What causes this?=C2=A0

Also, is there any document that explains what all the nodetool ab= breviations (UN, UL) stand for?=C2=A0

On Mon, Oct 27, 2014 at 5:46 AM, jivko = donev <jivko_d88@yahoo.com> wrote:
As I see the state=C2=A0162.243.109.94 is U= L(Up/Leaving) so maybe this is causing the problem.
<= div>


On Sunday, October 26, 2014 11:57 PM, Tim Dunphy &= lt;bluethundr@gma= il.com> wrote:


Hey all,

=C2=A0I'm trying to decommission a node.= =C2=A0

=C2=A0First I'm getting a status:
=

[root@beta-new:/usr/local] #nodetool status
<= div>Note: Ownership information does not include topology; for complete inf= ormation, specify a keyspace
Datacenter: datacenter1
= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=
Status=3DUp/Down
|/ State=3DNormal/Leaving/Joining/Moving
-- =C2=A0Address =C2=A0 =C2=A0 =C2=A0 =C2=A0 Load =C2=A0 =C2=A0 =C2= =A0 Tokens =C2=A0Owns =C2=A0 =C2=A0Host ID =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 R= ack
UN =C2=A0162.243.86.41 =C2=A0 1.08 MB =C2=A0 =C2=A01 =C2=A0 = =C2=A0 =C2=A0 0.1% =C2=A0 =C2=A0e945f3b5-2e3e-4a20-b1bd-e30c474a7634 =C2=A0= rack1
UL =C2=A0162.243.109.94 =C2=A01.28 MB =C2=A0 =C2=A0256 =C2= =A0 =C2=A0 99.9% =C2=A0 fd2f76ae-8dcf-4e93-a37f-bf1e9088696e =C2=A0rack1


But when I try to decommission the no= de I get this message:

[root@beta-new:/usr/lo= cal] #nodetool -h 162.243.86.41 decommission
nodetool: Failed to = connect to '162.243.86.41:7199' - NoSuchObjectException: 'no= such object in table'.

Yet I can telnet= to that host on that port just fine:

[root@b= eta-new:/usr/local] #telnet 162.243.86.41 7199
Trying 162.243.86.= 41...
Connected to 162.243.86.41.
Escape character is &= #39;^]'.


And I have verif= ied that cassandra is running and accessible via cqlsh on the other machine= .=C2=A0

What could be going wrong?=C2=A0

=
Thanks
Tim


--
GPG me!= !

gpg --keyserver pool.sks-keyservers.net --recv-keys F18619= 7B



<= /div>


--
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F1= 86197B





--
= GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

--f46d0444ec9914247e050667e299--