incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Bailey <n...@datastax.com>
Subject Re: Strange Connection error of nodetool
Date Wed, 22 Jun 2011 16:46:54 GMT
This is almost certainly caused by the weird connection process JMX
uses. JMX actually uses a 2 connection process, the second connection
is determined by the 'JVM_OPTS="$JVM_OPTS
-Djava.rmi.server.hostname=<public name>"' setting in your
cassandra-env.sh configuration file.

By default that setting is commented out and the hostname will default
to whatever the local machine thinks it's hostname is. You should be
able to fix this by uncommenting that setting in cassandra-env.sh and
using the correct hostname for that setting.

On Wed, Jun 22, 2011 at 4:02 AM, 박상길 <hayarobipark@gmail.com> wrote:
> Hi.
>
> I'm running 5 cassandra nodes. Say, the addresses are 112.234.123.111 ~ 112.234.123.115;
the real address is different.
> When I run nodetool, the one node of address 112.234.123.112 has failed to connect. Showing
error message like this.
>
> iPark:~ hayarobi$ nodetool --host 112.234.123.112 ring
> Error connection to remote JMX agent!
> java.rmi.ConnectException: Connection refused to host: 122.234.123.112; nested exception
is:
>
> The host to connect address differ! I had tried to query 112.* but, the nodetool tried
to connect 122.*. It happened just one machine. All other machines works fine.
> And I can connect to 112.234.123.112 by cassandra-cli or other tools using other port
(such as 22 of ssh, 80 of http). It has trouble only on nodetool.
>
> Does anyone has an idea?
>
> I'll paste the full stack trace below.
>
> iPark:~ hayarobi$ nodetool --host 112.234.123.111 ring
> Address         Status State   Load            Owns    Token
>                                                       136112946768375000000000000000000000000
> 112.234.123.111  Up     Normal  725.01 KB       20.00%  0
> 112.234.123.112  Up     Normal  725.93 KB       20.00%  34028236692093800000000000000000000000
> 112.234.123.113  Up     Normal  728.2 KB        20.00%  68056473384187700000000000000000000000
> 112.234.123.114  Up     Normal  713.1 KB        20.00%  102084710076282000000000000000000000000
> 112.234.123.115  Up     Normal  722.67 KB       20.00%  136112946768375000000000000000000000000
> iPark:~ hayarobi$ nodetool --host 112.234.123.115 ring
> Address         Status State   Load            Owns    Token
>                                                       136112946768375000000000000000000000000
> 112.234.123.111  Up     Normal  725.01 KB       20.00%  0
> 112.234.123.112  Up     Normal  725.93 KB       20.00%  34028236692093800000000000000000000000
> 112.234.123.113  Up     Normal  728.2 KB        20.00%  68056473384187700000000000000000000000
> 112.234.123.114  Up     Normal  713.1 KB        20.00%  102084710076282000000000000000000000000
> 112.234.123.115  Up     Normal  722.67 KB       20.00%  136112946768375000000000000000000000000
> iPark:~ hayarobi$ nodetool --host 112.234.123.114 ring
> Address         Status State   Load            Owns    Token
>                                                       136112946768375000000000000000000000000
> 112.234.123.111  Up     Normal  725.01 KB       20.00%  0
> 112.234.123.112  Up     Normal  725.93 KB       20.00%  34028236692093800000000000000000000000
> 112.234.123.113  Up     Normal  728.2 KB        20.00%  68056473384187700000000000000000000000
> 112.234.123.114  Up     Normal  713.1 KB        20.00%  102084710076282000000000000000000000000
> 112.234.123.115  Up     Normal  722.67 KB       20.00%  136112946768375000000000000000000000000
> iPark:~ hayarobi$ nodetool --host 112.234.123.112 ring
> Error connection to remote JMX agent!
> java.rmi.ConnectException: Connection refused to host: 122.234.123.112; nested exception
is:
>        java.net.ConnectException: Connection refused
>        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
>        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
>        at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
>        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
>        at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
>        at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2327)
>        at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:279)
>        at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
>        at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:137)
>        at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:107)
>        at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:511)
> Caused by: java.net.ConnectException: Connection refused
>        at java.net.PlainSocketImpl.socketConnect(Native Method)
>        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
>        at java.net.Socket.connect(Socket.java:529)
>        at java.net.Socket.connect(Socket.java:478)
>        at java.net.Socket.<init>(Socket.java:375)
>        at java.net.Socket.<init>(Socket.java:189)
>        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
>        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
>        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
>        ... 10 more
>
>

Mime
View raw message