incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ran Tavory <ran...@gmail.com>
Subject Re: ConcurrentModificationException in gossiper while decommissioning another node
Date Thu, 20 May 2010 06:31:07 GMT
that sounds like it, thanks

On Tue, May 18, 2010 at 3:53 PM, roger schildmeijer
<schildmeijer@gmail.com>wrote:

> This is hopefully fixed in trunk (CASSANDRA-757 (revision 938597));
> "Replace synchronization in Gossiper with concurrent data structures and
> volatile fields."
>
> // Roger Schildmeijer
>
>
> On Tue, May 18, 2010 at 1:55 PM, Ran Tavory <rantav@gmail.com> wrote:
>
>> While the node 192.168.252.61 was in the process of decommissioning I see
>> this error in two other nodes:
>>
>>  INFO [Timer-1] 2010-05-18 06:01:12,048 Gossiper.java (line 179)
>> InetAddress /192.168.252.62 is now dead.
>>  INFO [GMFD:1] 2010-05-18 06:04:00,189 Gossiper.java (line 568)
>> InetAddress /192.168.252.62 is now UP
>>  INFO [Timer-1] 2010-05-18 06:11:45,311 Gossiper.java (line 401) FatClient
>> /192.168.252.61 has been silent for 3600000ms, removing from gossip
>> ERROR [Timer-1] 2010-05-18 06:11:45,315 CassandraDaemon.java (line 88)
>> Fatal exception in thread Thread[Timer-1,5,main]
>> java.lang.RuntimeException: java.util.ConcurrentModificationException
>>         at
>> org.apache.cassandra.gms.Gossiper$GossipTimerTask.run(Gossiper.java:97)
>>         at java.util.TimerThread.mainLoop(Timer.java:512)
>>         at java.util.TimerThread.run(Timer.java:462)
>> Caused by: java.util.ConcurrentModificationException
>>         at java.util.Hashtable$Enumerator.next(Hashtable.java:1031)
>>         at
>> org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:382)
>>         at
>> org.apache.cassandra.gms.Gossiper$GossipTimerTask.run(Gossiper.java:91)
>>         ... 2 more
>>
>>
>> .61 is the decommissioned node. .62 was under load (streams transferred to
>> it from .61)
>>
>> I simply ran nodetool decommission on the 61 node and then (after an hour,
>> I guess) I saw this error in two other live nodes.
>>
>> Does this ring any bell? It's either a bug, or that I wasn't
>> running decommission correctly...
>>
>
>

Mime
View raw message