cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brandon Williams <dri...@gmail.com>
Subject Re: Nodes failed to bootstrap, no nodetool info but system.peer populated.
Date Mon, 11 May 2015 14:29:23 GMT
https://issues.apache.org/jira/browse/CASSANDRA-9180

On Mon, May 11, 2015 at 4:17 AM, Carlos Rolo <rolo@pythian.com> wrote:

> Hi all,
>
> I just wanted to know if this should be worth filling a bug or not
> (Couldn't find any similar).
>
> I have a 3 node cluster (2.0.14). Decided to add 3 new ones. 2 failed
> because of hardware failure (virtualized environment).
>
> The process was automated, so what was supposed to happen was:
>
> - Node 4 joins
> - wait until status is UN and then 2min more
> - Node 5 joins
> - wait until status is UN and then 2min more
> - Node 6 joins
> - wait until status is UN and then 2min more
>
> What happened:
> - Node 4 joins
> - Wait...
> - Node 5 joins
> - VM fails while node is starting.
> - VM 6 starts, no node with UN, waits 2min
> - Node 6 joins
> - VM fails while node is starting.
>
> After this, nodetool reports 4 nodes all UN
> While trying an application (Datastax Java Driver 2.1) the debug log
> reports that it tries to connect to Node 5 and 6 and fails.
>
> Checking system.peers table, I see both nodes there. So I tried "nodetool
> removenode <ID>" with the IDs in the table.
>
> It blows up with the following exception:
> Exception in thread "main" java.lang.UnsupportedOperationException: Host ID
> not found.
>
> Then I decided to do the following:
> DELETE from peers where ID in (ID1, ID2);
>
> All good, cluster still happy and driver not complaining anymore.
> Is this expected behavior?
>
>
>
> Regards,
>
> Carlos Juzarte Rolo
> Cassandra Consultant
>
> Pythian - Love your data
>
> rolo@pythian | Twitter: cjrolo | Linkedin: *
> linkedin.com/in/carlosjuzarterolo
> <http://linkedin.com/in/carlosjuzarterolo>*
> Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
> www.pythian.com
>
> --
>
>
> --
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message