cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Shuler (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-8306) exception in nodetool enablebinary
Date Mon, 17 Nov 2014 22:14:34 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael Shuler resolved CASSANDRA-8306.
---------------------------------------
    Resolution: Incomplete

So it sounds like bootstrapping failed for some reason, previously, and starting a fresh bootstrap
completed successfully. This can happen for various reasons, and when it does, the standard
procedure is to start the bootstrap over. Without the failed bootstrap logs to look at, it's
difficult to guess.

I'm glad that you were able to work through the problem!

> exception in nodetool enablebinary
> ----------------------------------
>
>                 Key: CASSANDRA-8306
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8306
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Rafał Furmański
>         Attachments: system.log.zip
>
>
> I was trying to add new node (db4) to existing cluster - with no luck. I can't see any
errors in system.log. nodetool status shows, that node is joining into cluster (for many hours).
Attaching error and cluster info:
> {code}
> root@db4:~# nodetool enablebinary
> error: Error starting native transport: null
> -- StackTrace --
> java.lang.RuntimeException: Error starting native transport: null
> 	at org.apache.cassandra.service.StorageService.startNativeTransport(StorageService.java:350)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
> 	at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
> 	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> 	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> 	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> 	at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> 	at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> 	at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
> 	at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
> 	at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
> 	at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
> 	at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
> 	at sun.rmi.transport.Transport$1.run(Transport.java:177)
> 	at sun.rmi.transport.Transport$1.run(Transport.java:174)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
> 	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> {code}
> root@db4:~# nodetool describecluster
> Cluster Information:
> 	Name: Production Cluster
> 	Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
> 	Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> 	Schema versions:
> 		b7e98bb9-717f-3f59-bac4-84bc19544e90: [10.195.15.163, 10.195.15.162, 10.195.15.167,
10.195.15.166]
> {code}
> {code}
> root@db4:~# nodetool status
> Datacenter: Ashburn
> ===================
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address        Load       Tokens  Owns    Host ID                               Rack
> UN  10.195.15.163  12.05 GB   256     ?       0a9f478c-80b5-4c15-8b2e-e27df6684c69  RAC1
> UN  10.195.15.162  12.8 GB    256     ?       c18d2218-ef84-4165-9c3a-05f592f512e9  RAC1
> UJ  10.195.15.167  18.61 GB   256     ?       0d3999d9-1e33-4407-bbbd-10cf0a93b3ba  RAC1
> UN  10.195.15.166  13.67 GB   256     ?       df8df3b7-da17-48de-8cf6-1c718dc2fde8  RAC1
> {code}
> I can't even connect to cassandra using cqlsh:
> {code}
> root@db4:~# cqlsh 10.195.15.167
> Connection error: ('Unable to connect to any servers', {'10.195.15.167': error(111, "Tried
connecting to [('10.195.15.167', 9042)]. Last error: Connection refused")})
> {code}
> In OpsCenter I can see that node has status: Active - Joining and Gossip is active. Any
ideas what could be wrong with this node?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message