Mailing-List: contact dev-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CALcD3Pv+Wx_twmftzq0S-g42_pD-r43fFeKz42hX7tPawLvL5g@mail.gmail.com>
References: 
 <CALcD3Pv+Wx_twmftzq0S-g42_pD-r43fFeKz42hX7tPawLvL5g@mail.gmail.com>
From: Brandon Williams <driftx@gmail.com>
Date: Mon, 11 May 2015 09:29:23 -0500
Message-ID: 
 <CAAafH9T7g6V-CxsQdrgGYg3XPZkcHUPr8U1LQ_i45O+uU8p-fA@mail.gmail.com>
Subject: Re: Nodes failed to bootstrap,
 no nodetool info but system.peer populated.
To: "dev@cassandra.apache.org" <dev@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=001a113cc9fe8249ae0515cf359f

--001a113cc9fe8249ae0515cf359f
Content-Type: text/plain; charset=UTF-8

https://issues.apache.org/jira/browse/CASSANDRA-9180

On Mon, May 11, 2015 at 4:17 AM, Carlos Rolo <rolo@pythian.com> wrote:

> Hi all,
>
> I just wanted to know if this should be worth filling a bug or not
> (Couldn't find any similar).
>
> I have a 3 node cluster (2.0.14). Decided to add 3 new ones. 2 failed
> because of hardware failure (virtualized environment).
>
> The process was automated, so what was supposed to happen was:
>
> - Node 4 joins
> - wait until status is UN and then 2min more
> - Node 5 joins
> - wait until status is UN and then 2min more
> - Node 6 joins
> - wait until status is UN and then 2min more
>
> What happened:
> - Node 4 joins
> - Wait...
> - Node 5 joins
> - VM fails while node is starting.
> - VM 6 starts, no node with UN, waits 2min
> - Node 6 joins
> - VM fails while node is starting.
>
> After this, nodetool reports 4 nodes all UN
> While trying an application (Datastax Java Driver 2.1) the debug log
> reports that it tries to connect to Node 5 and 6 and fails.
>
> Checking system.peers table, I see both nodes there. So I tried "nodetool
> removenode <ID>" with the IDs in the table.
>
> It blows up with the following exception:
> Exception in thread "main" java.lang.UnsupportedOperationException: Host ID
> not found.
>
> Then I decided to do the following:
> DELETE from peers where ID in (ID1, ID2);
>
> All good, cluster still happy and driver not complaining anymore.
> Is this expected behavior?
>
>
>
> Regards,
>
> Carlos Juzarte Rolo
> Cassandra Consultant
>
> Pythian - Love your data
>
> rolo@pythian | Twitter: cjrolo | Linkedin: *
> linkedin.com/in/carlosjuzarterolo
> <http://linkedin.com/in/carlosjuzarterolo>*
> Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
> www.pythian.com
>
> --
>
>
> --
>
>
>
>

--001a113cc9fe8249ae0515cf359f--