cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peer, Oded" <Oded.P...@rsa.com>
Subject RE: Can't connect to Cassandra server
Date Mon, 20 Jul 2015 07:18:52 GMT
Cassandra does some housekeeping before it starts accepting requests from clients.
For instance, as you can see from the log file, it replays commit log.
When the node is ready to accept requests from clients it logs a message to the log file similar
to “Starting listening for CQL clients on localhost/127.0.0.1:9042...”
This log message does not appear in your log file. It appears the node is still doing housekeeping.
If you look in your log file you see a lot of “Enqueuing flush of XXX” near the end of
the log file, then the StatusLogger prints the tasks status. It shows you have 69 pending
flushes and only 1 completed.
Another finding in the log is the GCInspector message showing that GC occurred and barely
removed data from your Old Gen (from 3GB to 2.7 GB) in 11 seconds.

What are your JVM heap settings?
Have you changed the default configuration of flushing parameters or memTable parameters?



From: erickramirezonline@gmail.com [mailto:erickramirezonline@gmail.com] On Behalf Of Erick
Ramirez
Sent: Monday, July 20, 2015 4:08 AM
To: user@cassandra.apache.org
Subject: Re: Can't connect to Cassandra server

Hello, Chamila.

From the information you have supplied so far, it is not clear whether Cassandra is in fact
running.

Please provide the output of the following:

$ netstat -a -n | grep LISTEN
$ sudo lsof -i -n | grep LISTEN | grep java

It would also be good if you could provide output.log

Cheers,
Erick

Erick Ramirez
About Me about.me/erickramirezonline<http://about.me/erickramirezonline>

Make a difference today!
* Reduce your carbon footprint<http://on.mash.to/1vZL7fX>
* Give back to the community<http://www.govolunteer.com.au>
* Write free software<http://www.opensource.org>


On Sun, Jul 19, 2015 at 11:22 PM, Chamila Wijayarathna <cdwijayarathna@gmail.com<mailto:cdwijayarathna@gmail.com>>
wrote:
Hi Peer,

https://gist.githubusercontent.com/cdwijayarathna/a14586a9e39a943f89a0/raw/system%20log This
is the log of the last time I started the server, I couldn't found any error there.

Thanks

On Sun, Jul 19, 2015 at 5:56 PM, Peer, Oded <Oded.Peer@rsa.com<mailto:Oded.Peer@rsa.com>>
wrote:
Try looking in the log file (/var/log/cassandra/system.log) for errors that prevent your node
from starting.


From: Chamila Wijayarathna [mailto:cdwijayarathna@gmail.com<mailto:cdwijayarathna@gmail.com>]
Sent: Sunday, July 19, 2015 2:29 PM

To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Can't connect to Cassandra server

Hi,

I'm getting following error when running nodetool status.

maduranga@ubuntu:/etc/cassandra$ nodetool status
error: No nodes present in the cluster. Has this node finished starting up?
-- StackTrace --
java.lang.RuntimeException: No nodes present in the cluster. Has this node finished starting
up?
            at org.apache.cassandra.dht.Murmur3Partitioner.describeOwnership(Murmur3Partitioner.java:129)
            at org.apache.cassandra.service.StorageService.getOwnership(StorageService.java:3702)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
            at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
            at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
            at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
            at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
            at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
            at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
            at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
            at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
            at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)
            at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
            at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
            at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
            at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:657)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
            at sun.rmi.transport.Transport$1.run(Transport.java:177)
            at sun.rmi.transport.Transport$1.run(Transport.java:174)
            at java.security.AccessController.doPrivileged(Native Method)
            at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
            at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
            at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
            at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
            at java.lang.Thread.run(Thread.java:745)

What is the reason for this? How can I fix this?

Thank You!

On Sun, Jul 19, 2015 at 4:52 PM, Peer, Oded <Oded.Peer@rsa.com<mailto:Oded.Peer@rsa.com>>
wrote:
Are you sure your node is up? Do you get a result when running “nodetool –h 192.248.15.219
status”?

From: Chamila Wijayarathna [mailto:cdwijayarathna@gmail.com<mailto:cdwijayarathna@gmail.com>]
Sent: Sunday, July 19, 2015 1:53 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Can't connect to Cassandra server

Hi Umang,

Tried your suggestion, but still getting the same error.

Thanks.

On Sun, Jul 19, 2015 at 3:12 PM, Umang Shah <shahumang4@gmail.com<mailto:shahumang4@gmail.com>>
wrote:
You also have to change the same IP which is 192.248.15.219 for seeds inside cassandra.yaml
file.

then try to connect, it will work.

Thanks,
Umang Shah

On Sun, Jul 19, 2015 at 1:52 AM, Chamila Wijayarathna <cdwijayarathna@gmail.com<mailto:cdwijayarathna@gmail.com>>
wrote:
Hi Ajay,

I tried that also, but still getting the same result.

On Sun, Jul 19, 2015 at 2:08 PM, Ajay <ajay.garga@gmail.com<mailto:ajay.garga@gmail.com>>
wrote:
Try with the correct IP address as below:

cqlsh 192.248.15.219 -u sinmin -p xxxxxx
CQL documentation - http://docs.datastax.com/en/cql/3.0/cql/cql_reference/cqlsh.html

On Sun, Jul 19, 2015 at 2:00 PM, Chamila Wijayarathna <cdwijayarathna@gmail.com<mailto:cdwijayarathna@gmail.com>>
wrote:
Hello all,

After starting cassandra, I tried to connect to cassandra from cqlsh and java, but it fails
to do so.

Following is the error I get while trying to connect to cqlsh.

cqlsh -u sinmin -p xxxxxx
Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting
to [('127.0.0.1', 9042)]. Last error: Connection refused")})

I have set listen_address and rpc_address in cassandra.yaml to the ip address of server address
like follows.

listen_address:192.248.15.219
rpc_address:192.248.15.219

Following is what I found from cassandra system.log.
https://gist.githubusercontent.com/cdwijayarathna/a14586a9e39a943f89a0/raw/system%20log

Following is the netstat result I got.

maduranga@ubuntu:/var/log/cassandra$ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 ubuntu:ssh              103.21.166.35:54417<http://103.21.166.35:54417>
    ESTABLISHED
tcp        0      0 ubuntu:1522             ubuntu:30820            ESTABLISHED
tcp        0      0 ubuntu:30820            ubuntu:1522             ESTABLISHED
tcp        0    256 ubuntu:ssh              175.157.41.209:42435<http://175.157.41.209:42435>
   ESTABLISHED
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  9      [ ]         DGRAM                    7936     /dev/log
unix  3      [ ]         STREAM     CONNECTED     11737
unix  3      [ ]         STREAM     CONNECTED     11736
unix  3      [ ]         STREAM     CONNECTED     10949    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10948
unix  2      [ ]         DGRAM                    10947
unix  2      [ ]         STREAM     CONNECTED     10801
unix  3      [ ]         STREAM     CONNECTED     10641
unix  3      [ ]         STREAM     CONNECTED     10640
unix  3      [ ]         STREAM     CONNECTED     10444    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10443
unix  3      [ ]         STREAM     CONNECTED     10437    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10436
unix  3      [ ]         STREAM     CONNECTED     10430    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10429
unix  2      [ ]         DGRAM                    10424
unix  3      [ ]         STREAM     CONNECTED     10422    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10421
unix  2      [ ]         DGRAM                    10420
unix  2      [ ]         STREAM     CONNECTED     10215
unix  2      [ ]         STREAM     CONNECTED     10296
unix  2      [ ]         STREAM     CONNECTED     9988
unix  2      [ ]         DGRAM                    9520
unix  3      [ ]         STREAM     CONNECTED     8769     /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     8768
unix  2      [ ]         DGRAM                    8753
unix  2      [ ]         DGRAM                    9422
unix  3      [ ]         STREAM     CONNECTED     7000     @/com/ubuntu/upstart
unix  3      [ ]         STREAM     CONNECTED     8485
unix  2      [ ]         DGRAM                    7947
unix  3      [ ]         STREAM     CONNECTED     6712     /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     6711
unix  3      [ ]         STREAM     CONNECTED     7760     /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     7759
unix  3      [ ]         STREAM     CONNECTED     7754
unix  3      [ ]         STREAM     CONNECTED     7753
unix  3      [ ]         DGRAM                    7661
unix  3      [ ]         DGRAM                    7660
unix  3      [ ]         STREAM     CONNECTED     6490     @/com/ubuntu/upstart
unix  3      [ ]         STREAM     CONNECTED     6475

What is the issue here? Why I can't connect to Cassandra server? How can I fix this?

Thank You!

--
Chamila Dilshan Wijayarathna,
Software Engineer
Mobile:(+94)788193620<tel:%28%2B94%29788193620>
WSO2 Inc., http://wso2.com/





--
Chamila Dilshan Wijayarathna,
Software Engineer
Mobile:(+94)788193620<tel:%28%2B94%29788193620>
WSO2 Inc., http://wso2.com/




--
Regards,
Umang Shah
+919886829019<tel:%2B919886829019>



--
Chamila Dilshan Wijayarathna,
Software Engineer
Mobile:(+94)788193620<tel:%28%2B94%29788193620>
WSO2 Inc., http://wso2.com/




--
Chamila Dilshan Wijayarathna,
Software Engineer
Mobile:(+94)788193620<tel:%28%2B94%29788193620>
WSO2 Inc., http://wso2.com/




--
Chamila Dilshan Wijayarathna,
Software Engineer
Mobile:(+94)788193620
WSO2 Inc., http://wso2.com/


Mime
View raw message