cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Failbus (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
Date Mon, 02 Nov 2015 17:50:28 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985589#comment-14985589
] 

Kenneth Failbus edited comment on CASSANDRA-8072 at 11/2/15 5:50 PM:
---------------------------------------------------------------------

Upon enabling trace on the one seed node and re-bootstrapping the new node, I got the following
exception on the node that was bootstrapping.
{code}
2015-11-02 17:34:52,150 [ACCEPT-/10.22.168.53] DEBUG MessagingService Error reading the socket
Socket[addr=/10.xx.xx.xx,port=46678,localport=10xxx]
java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at java.io.InputStream.read(InputStream.java:101)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:81)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.cassandra.net.MessagingService$SocketThread.run(MessagingService.java:916)
{code}

And then usual exception that this ticket is mentioning about
{code}
2015-11-02 17:34:55,526 [EXPIRING-MAP-REAPER:1] TRACE ExpiringMap Expired 0 entries
2015-11-02 17:35:00,526 [EXPIRING-MAP-REAPER:1] TRACE ExpiringMap Expired 0 entries
2015-11-02 17:35:05,527 [EXPIRING-MAP-REAPER:1] TRACE ExpiringMap Expired 0 entries
2015-11-02 17:35:10,527 [EXPIRING-MAP-REAPER:1] TRACE ExpiringMap Expired 0 entries
2015-11-02 17:35:11,982 [main] ERROR CassandraDaemon Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
        at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1296)
        at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:457)
        at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:671)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:623)
        at org.apache.cassandra.service.StorageService.initServer(StorageService.java:515)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:437)
        at com.datastax.bdp.server.DseDaemon.setup(DseDaemon.java:423)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
        at com.datastax.bdp.server.DseDaemon.main(DseDaemon.java:641)
2015-11-02 17:35:11,986 [Thread-7] INFO DseDaemon DSE shutting down...
2015-11-02 17:35:11,987 [StorageServiceShutdownHook] WARN Gossiper No local state or state
is in silent shutdown, not announcing shutdown
2015-11-02 17:35:11,987 [StorageServiceShutdownHook] INFO MessagingService Waiting for messaging
service to quiesce
2015-11-02 17:35:11,987 [StorageServiceShutdownHook] DEBUG MessagingService Closing accept()
thread
2015-11-02 17:35:11,988 [ACCEPT-/10.22.168.53] DEBUG MessagingService Asynchronous close seen
by server thread
2015-11-02 17:35:11,988 [ACCEPT-/10.22.168.53] INFO MessagingService MessagingService has
terminated the accept() thread
2015-11-02 17:35:12,068 [Thread-7] ERROR CassandraDaemon Exception in thread Thread[Thread-7,5,main]
{code}


was (Author: kenfailbus):
Upon enabling trace on the one seed node and re-bootstrapping the new node, I got the following
exception on the node that was bootstrapping.
{code}
2015-11-02 17:34:52,150 [ACCEPT-/10.22.168.53] DEBUG MessagingService Error reading the socket
Socket[addr=/10.xx.xx.xx,port=46678,localport=10xxx]
java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:229)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at java.io.InputStream.read(InputStream.java:101)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:81)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.cassandra.net.MessagingService$SocketThread.run(MessagingService.java:916)
{code}

> Exception during startup: Unable to gossip with any seeds
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-8072
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Ryan Springer
>            Assignee: Stefania
>             Fix For: 2.1.x
>
>         Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2,
cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, casandra-system-log-with-assert-patch.log, screenshot-1.png,
trace_logs.tar.bz2
>
>
> When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either
ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*.  The
error in the /var/log/cassandra/system.log is:
> ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered
during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
>         at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200)
>         at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444)
>         at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655)
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609)
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502)
>         at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
>         at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
>         at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279)
Announcing shutdown
>  INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line
701) Waiting for messaging service to quiesce
>  INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line
941) MessagingService has terminated the accept() thread
> This errors does not always occur when provisioning a 2-node cluster, but probably around
half of the time on only one of the nodes.  I haven't been able to reproduce this error with
DSC 2.0.9, and there have been no code or definition file changes in Opscenter.
> I can reproduce locally with the above steps.  I'm happy to test any proposed fixes
since I'm the only person able to reproduce reliably so far.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message