Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B08A711CD3 for ; Sun, 1 Jun 2014 04:18:02 +0000 (UTC) Received: (qmail 39435 invoked by uid 500); 1 Jun 2014 04:18:00 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 39395 invoked by uid 500); 1 Jun 2014 04:18:00 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 39387 invoked by uid 99); 1 Jun 2014 04:18:00 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2014 04:18:00 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of bluethundr@gmail.com designates 74.125.82.50 as permitted sender) Received: from [74.125.82.50] (HELO mail-wg0-f50.google.com) (74.125.82.50) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 01 Jun 2014 04:17:56 +0000 Received: by mail-wg0-f50.google.com with SMTP id x12so3599778wgg.33 for ; Sat, 31 May 2014 21:17:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Kh2gKX6TZ4ub9P+zajX5HTXOodrYQAveIAMrVF4R64M=; b=of+4zHOIRCvKXuzsTshKDECAgjL9Trr0FurOL8fCNAmw0roG7v771T5yh0bkHKJjBS 5Tx5q1dWX4Rnhuln6YoPkpUt2lPuedV5IbgVoztoZEDRzE9AoPqF4jVcGqImN/2vXiV9 mrxTYNJL5Pd3gwW5uR8T5D1vYluWO0RtfNgmc1cV0mNNy1QMujlKxJvPgwY5TwejOOuA 0WzUExo3NsqcjepnAO5luhKaMGzZMO5vP3idHWPkmMScpcU7H/1DQKd5MzH74QEz3Jat xBjR6Pewq5dN2mpRafx7HX9B9/joR7yoFEZ8IjqlnI+cogjjGou1Fdc/tEuViOFzTK2w tp2g== MIME-Version: 1.0 X-Received: by 10.180.11.9 with SMTP id m9mr11256795wib.51.1401596255437; Sat, 31 May 2014 21:17:35 -0700 (PDT) Received: by 10.216.77.131 with HTTP; Sat, 31 May 2014 21:17:35 -0700 (PDT) In-Reply-To: References: Date: Sun, 1 Jun 2014 00:17:35 -0400 Message-ID: Subject: Re: decommission dead node without token From: Tim Dunphy To: "user@cassandra.apache.org" Content-Type: multipart/alternative; boundary=001a11c258dcecdd3404fabe8e44 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c258dcecdd3404fabe8e44 Content-Type: text/plain; charset=UTF-8 sorry for the barrage of questions.... but I'm having trouble with the nodetool remove phase. [root@cassandra01 ~]# nodetool removetoken -h 10.10.1.68 -9208584805646615844 Gives me a unix usage message. As in: [root@cassandra01 ~]# nodetool removetoken -h 107.170.178.68 -9208584805646615844 | head -5 Unrecognized option: -9208584805646615844 usage: java org.apache.cassandra.tools.NodeCmd --host -a,--include-all-sstables includes sstables that are already on the most recent version during upgradesstables -c,--compact print histograms in a more compact format I think the problem is that the bash environment is getting confused by the minus sign in front of the token. But that is the token and the minus sign is integral to it. I've even tried quoting it with both single quotes and double quotes to no avail. I still get the same usage message. Can someone please help with the right syntax here? Thanks Tim On Sun, Jun 1, 2014 at 12:01 AM, Tim Dunphy wrote: > Just wondering.. might I be able to retried the 'lost' intital_token of > the node I want to remove by going: > > [root@beta-new:~] #nodetool ring | grep 10.10.1.102 | head -1 > 107.170.175.102 rack1 Down Normal 123.41 KB 32.19% > -9208584805646615844 > > Then > > 1) place that token in the cassandra.yaml of a new node with a new IP as > -9208584805646615844 -1 > 2) set auto_bootstrap in the cassandra.yaml > 3) once the node boots run 'nodetool removetoken -9208584805646615844' > 4) run nodetool cleanup > > Would this be the way to get out of this bind? Just want to be sure I > understand the process. > > Thanks > Tim > > > On Sat, May 31, 2014 at 10:38 PM, Tim Dunphy wrote: > >> hey guys, >> >> I have two dead nodes in my ring. >> >> Status=Up/Down >> >> |/ State=Normal/Leaving/Joining/Moving >> >> -- Address Load Tokens Owns Host ID >> Rack >> >> DN 10.10.1.102 123.41 KB 256 32.2% >> ddcafc75-24ed-4c6a-99bb-afe3dd551a9c rack1 >> >> UN 10.10.1.94 170.81 KB 256 35.4% >> fd2f76ae-8dcf-4e93-a37f-bf1e9088696e rack1 >> >> DN 10..10.10.64 ? 256 32.5% >> f2a48fc7-a362-43f5-9061-4bb3739fdeaf rack1 >> >> >> Unfortunately I am early in my learning curve and failed to preserve the >> initial_token for each of the nodes that were taken out of service. As >> things stand now I cannot do a nodetool repair on the main keyspace because >> of the dead nodes. >> >> Is there any way to decommission the dead nodes if you no longer have >> their initial_token values? >> >> >> Thanks >> >> Tim >> >> -- >> GPG me!! >> >> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B >> >> > > > -- > GPG me!! > > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B > > -- GPG me!! gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B --001a11c258dcecdd3404fabe8e44 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
sorry for the barrage of questions.... but I'm having = trouble with the nodetool remove phase.=C2=A0

[root@cass= andra01 ~]# nodetool removetoken -h 10.10.1.68 -9208584805646615844

Gives me a unix usage message. As in:

[root@cassandra01 ~]# nodetool removetoken -h 107.170.178.68 -92085= 84805646615844 | head -5
Unrecognized option: -920858480564661584= 4
usage: java org.apache.cassandra.tools.NodeCmd --host <arg> <= command>

=C2=A0-a,--include-all-sstables =C2=A0= includes sstables that are already on the
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0most recent version during upgradesstables
=C2=A0-c,--compact =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0print histograms in a more compact format

<= div>I think the problem is that the bash environment is getting confused by= the minus sign in front of the token. But that is the token and the minus = sign is integral to it. I've even tried quoting it with both single quo= tes and double quotes to no avail. I still get the same usage message.

Can someone please help with the right syntax here?

Thanks
Tim


On Sun, Jun 1, 2014 at 12:01 AM, T= im Dunphy <bluethundr@gmail.com> wrote:
Just wondering.. might I be= able to retried the 'lost' intital_token of the node I want to rem= ove by going:

[root@beta-new:~] #nodetool ring | grep 10.10.1.102 | h= ead -1
107.170.175.102 =C2=A0rack1 =C2=A0 =C2=A0 =C2=A0 Down =C2=A0 Normal = =C2=A0123.41 KB =C2=A0 =C2=A0 =C2=A0 32.19% =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0-9208584805646615844

Then

1) place that token in the cassandra.yaml of a new = node with a new IP as =C2=A0-9208584805646615844 -1=C2=A0
2) =C2=A0set auto_bootstrap in the cassandra.yaml
3) once th= e node boots run 'nodetool removetoken -9208584805646615844'
<= div>4) run nodetool cleanup

Would this be the way = to get out of this bind? Just want to be sure I understand the process.

Thanks
Tim


On Sat, May 31, 2014 at 10:38 PM, = Tim Dunphy <bluethundr@gmail.com> wrote:
hey guys,

=C2=A0I have two dead nodes in my ring.=C2= =A0

Status=3DUp/Down

|/ State=3DNormal/Leaving/Joining/Moving

--=C2=A0 Address=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Load =C2=A0 =C2=A0 = =C2=A0 Tokens=C2=A0 Owns =C2=A0 Host ID =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Rack<= /p>

DN=C2=A0 10.10.1.102=C2=A0 123.41 KB=C2=A0 256 =C2=A0 =C2=A0 32.2%=C2=A0= ddcafc75-24ed-4c6a-99bb-afe3dd551a9c=C2=A0 rack1

UN=C2=A0 10.10.1.94 =C2=A0 170.81 KB=C2=A0 256 =C2=A0 =C2=A0 35.4%=C2=A0= fd2f76ae-8dcf-4e93-a37f-bf1e9088696e=C2=A0 rack1

DN=C2=A0 10..10.10.64 =C2=A0 ?=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 256 =C2= =A0 =C2=A0 32.5%=C2=A0 f2a48fc7-a362-43f5-9061-4bb3739fdeaf=C2=A0 rack1

=


Unfortunately I am early in my learning curve and failed to p= reserve the initial_token for each of the nodes that were taken out of serv= ice. As things stand now I cannot do a nodetool repair on the main keyspace= because of the dead nodes.=C2=A0

Is there any way to decommission the dead nodes if you no longer have th= eir initial_token values?


Thanks

Tim


--
GPG me!!

gpg --keyserver pool.sks-keyserve= rs.net --recv-keys F186197B




--
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv= -keys F186197B




--
GPG me!!
=
gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

--001a11c258dcecdd3404fabe8e44--