cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From i...@4friends.od.ua
Subject Re: Node joining / unknown
Date Wed, 07 Mar 2012 21:23:23 GMT
just run "nodetool compactionstat" on other nodes.


-----Original Message-----
From: "R. Verlangen" <robin@us2.nl>
To: user@cassandra.apache.org
Sent: Wed, 07 Mar 2012 23:09
Subject: Re: Node joining / unknown

@Brandon: Thank you for the information. I'll do that next time.

@Igor: Any ways to find out whether that is the current state? And if so,
how to solve it?

2012/3/7 <igor@4friends.od.ua>

> Maybe it wait for verification compaction on other node?
>
>
>
>
>
> -----Original Message-----
> From: "R. Verlangen" <robin@us2.nl>
> To: user@cassandra.apache.org
> Sent: Wed, 07 Mar 2012 22:15
> Subject: Re: Node joining / unknown
>
> At this moment the node has joined the ring (after a restart: tried that
> before, but now it had finally result).
>
> When I try to run repair on the new node, the log says (the new node is
> NODE C):
>
> INFO [AntiEntropyStage:1] 2012-03-07 21:12:06,453 AntiEntropyService.java
> (line 190) [repair #cfcc12b0-6891-11e1-0000-70a329caccff] Received merkle
> tree for StorageMeta from NODE A
>  INFO [AntiEntropyStage:1] 2012-03-07 21:12:06,643 AntiEntropyService.java
> (line 190) [repair #cfcc12b0-6891-11e1-0000-70a329caccff] Received merkle
> tree for StorageMeta from NODE B
>
> And then doesn't do anything anymore. Tried it a couple of times again.
> It's just not starting.
>
> Results from netstats on NODE C:
>
> Mode: NORMAL
> Not sending any streams.
> Not receiving any streams.
> Pool Name                    Active   Pending      Completed
> Commands                        n/a         0              5
> Responses                       n/a        93           4296
>
>
> Any suggestions?
>
> Thank you!
>
> 2012/3/7 aaron morton <aaron@thelastpickle.com>
>
>> - When I try to remove the token, it says: Exception in thread "main"
>> java.lang.UnsupportedOperationException: Token not found.
>>
>> Am assuming you ran nodetool removetoken on a node other than the joining
>> node? What  did nodetool ring look like on that machine ?
>>
>> Take a look at nodetool netstats on the joining node to see if streaming
>> has failed. If it's dead then…
>>
>> 1) Try restarting the joining node and run nodetool repair on it
>> immediately. Note: am assuming QUOURM CL otherwise things may get
>> inconsistent.
>> or
>> 2) Stop the node. Try to get remove the token again from another node.
>> Node that removing a token will stream data around the place as well.
>>
>> Cheers
>>
>>   -----------------
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 7/03/2012, at 9:11 PM, R. Verlangen wrote:
>>
>> Hi there,
>>
>> I'm currently in a really weird situation.
>> - Nodetool ring says node X is joining (this already takes 12 hours, with
>> no activity)
>> - When I try to remove the token, it says: Exception in thread "main"
>> java.lang.UnsupportedOperationException: Token not found.
>> - Removetoken status = No token removals in process.
>>
>> How to get that node out of my cluster?
>>
>> With kind regards,
>> Robin Verlangen
>>
>>
>>
>

Mime
View raw message