cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roger schildmeijer <schildmei...@gmail.com>
Subject Re: ConcurrentModificationException in gossiper while decommissioning another node
Date Tue, 18 May 2010 12:53:17 GMT
This is hopefully fixed in trunk (CASSANDRA-757 (revision 938597)); "Replace
synchronization in Gossiper with concurrent data structures and volatile
fields."

// Roger Schildmeijer

On Tue, May 18, 2010 at 1:55 PM, Ran Tavory <rantav@gmail.com> wrote:

> While the node 192.168.252.61 was in the process of decommissioning I see
> this error in two other nodes:
>
>  INFO [Timer-1] 2010-05-18 06:01:12,048 Gossiper.java (line 179)
> InetAddress /192.168.252.62 is now dead.
>  INFO [GMFD:1] 2010-05-18 06:04:00,189 Gossiper.java (line 568) InetAddress
> /192.168.252.62 is now UP
>  INFO [Timer-1] 2010-05-18 06:11:45,311 Gossiper.java (line 401) FatClient
> /192.168.252.61 has been silent for 3600000ms, removing from gossip
> ERROR [Timer-1] 2010-05-18 06:11:45,315 CassandraDaemon.java (line 88)
> Fatal exception in thread Thread[Timer-1,5,main]
> java.lang.RuntimeException: java.util.ConcurrentModificationException
>         at
> org.apache.cassandra.gms.Gossiper$GossipTimerTask.run(Gossiper.java:97)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> Caused by: java.util.ConcurrentModificationException
>         at java.util.Hashtable$Enumerator.next(Hashtable.java:1031)
>         at
> org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:382)
>         at
> org.apache.cassandra.gms.Gossiper$GossipTimerTask.run(Gossiper.java:91)
>         ... 2 more
>
>
> .61 is the decommissioned node. .62 was under load (streams transferred to
> it from .61)
>
> I simply ran nodetool decommission on the 61 node and then (after an hour,
> I guess) I saw this error in two other live nodes.
>
> Does this ring any bell? It's either a bug, or that I wasn't
> running decommission correctly...
>

Mime
View raw message