cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom van den Berge <tom.vandenbe...@gmail.com>
Subject Re: Removed node is not completely removed
Date Thu, 15 Oct 2015 06:42:12 GMT
Thanks Sebastian, a restart solved the problem!


On Wed, Oct 14, 2015 at 3:46 PM, Sebastian Estevez <
sebastian.estevez@datastax.com> wrote:

> We still keep endpoints in memory. Not sure how you git to this state but
> try a rolling restart.
> On Oct 14, 2015 9:43 AM, "Tom van den Berge" <tom.vandenberge@gmail.com>
> wrote:
>
>> Thanks for that Michael, I did not know that. However, the node is not
>> listed in the system.peers table on any node, so it seems that the problem
>> is not in this table.
>>
>>
>>
>> On Wed, Oct 14, 2015 at 3:30 PM, Laing, Michael <
>> michael.laing@nytimes.com> wrote:
>>
>>> Remember that the system keyspace uses LocalStrategy: each node has its
>>> own set of system tables. -ml
>>>
>>> On Wed, Oct 14, 2015 at 9:17 AM, Tom van den Berge <
>>> tom.vandenberge@gmail.com> wrote:
>>>
>>>> Hi Carlos,
>>>>
>>>> I'm using 2.1.6. The mysterious node is not in the peers table. Any
>>>> other ideas?
>>>> One of my existing nodes is not present in the system.peers table,
>>>> though. Should I be worried?
>>>>
>>>> Regards,
>>>> Tom
>>>>
>>>> On Wed, Oct 14, 2015 at 2:27 PM, Carlos Rolo <rolo@pythian.com> wrote:
>>>>
>>>>> Check system.peers table to see if the IP is still there. If so edit
>>>>> the table and remove the offending IP.
>>>>>
>>>>> You are probably running into this:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-6053
>>>>>
>>>>> Regards,
>>>>>
>>>>> Carlos Juzarte Rolo
>>>>> Cassandra Consultant
>>>>>
>>>>> Pythian - Love your data
>>>>>
>>>>> rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
>>>>> <http://linkedin.com/in/carlosjuzarterolo>*
>>>>> Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
>>>>> www.pythian.com
>>>>>
>>>>> On Wed, Oct 14, 2015 at 12:26 PM, Tom van den Berge <
>>>>> tom.vandenberge@gmail.com> wrote:
>>>>>
>>>>>> I have removed a node with nodetool removenode, which completed ok.
>>>>>> Nodetool status does not list the node anymore.
>>>>>>
>>>>>> But since then, Im seeing messages in my other nodes log files
>>>>>> referring to the removed node:
>>>>>>
>>>>>>  INFO [GossipStage:38] 2015-10-14 11:18:26,322 Gossiper.java (line
>>>>>> 968) InetAddress /10.68.56.200 is now DOWN
>>>>>>  INFO [GossipStage:38] 2015-10-14 11:18:26,324 StorageService.java
>>>>>> (line 1891) Removing tokens [85070591730234615865843651857942052863]
for /
>>>>>> 10.68.56.200
>>>>>>
>>>>>>
>>>>>> These two messages appear every minute.
>>>>>> I've tried nodetool removenode again (Host ID not found) and
>>>>>> removenode force (no token removals in process).
>>>>>> The jmx unsafeAssassinateEndpoint gives a NullPointerException.
>>>>>>
>>>>>> What can I do to remove the node entirely?
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>

Mime
View raw message