cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lerh Chuan Low <l...@instaclustr.com>
Subject Re: Connection refused - 127.0.0.1-Gossip
Date Tue, 05 Dec 2017 22:14:36 GMT
I think as Jeff mentioned it sounds like a configuration issue, are you
sure you are using the same configmap/however it's being passed in and just
throwing out ideas, maybe the pods are behind a http proxy and you may have
forgotten to pass in the env vars?

On 6 December 2017 at 08:45, Jeff Jirsa <jjirsa@gmail.com> wrote:

> I don't have any k8 clusters to test with, but do you know how your yaml
> translates to cassandra.yaml ? What are the listen/broadcast addresses
> being set?
>
>
> On Tue, Dec 5, 2017 at 6:09 AM, Marek Kadek -T (mkadek - CONSOL PARTNERS
> LTD at Cisco) <mkadek@cisco.com> wrote:
>
>> We are experiencing following issues with Cassandra on our kubernetes
>> clusters:
>>
>> ```
>>
>> @ kubectl exec -it cassandra-cassandra-0 -- tail
>> /var/log/cassandra/debug.log
>>
>> DEBUG [MessagingService-Outgoing-localhost/127.0.0.1-Gossip] 2017-12-05
>> 09:02:06,560 OutboundTcpConnection.java:545 - Unable to connect to
>> localhost/127.0.0.1
>>
>> java.net.ConnectException: Connection refused
>>
>>                 at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_131]
>>
>>                 at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_131]
>>
>>                 at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_131]
>>
>>                 at sun.nio.ch.SocketChannelImpl.c
>> onnect(SocketChannelImpl.java:648) ~[na:1.8.0_131]
>>
>>                 at org.apache.cassandra.net.Outbo
>> undTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:146)
>> ~[apache-cassandra-3.11.0.jar:3.11.0]
>>
>>                 at org.apache.cassandra.net.Outbo
>> undTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:132)
>> ~[apache-cassandra-3.11.0.jar:3.11.0]
>>
>>                 at org.apache.cassandra.net.Outbo
>> undTcpConnection.connect(OutboundTcpConnection.java:433)
>> [apache-cassandra-3.11.0.jar:3.11.0]
>>
>>                 at org.apache.cassandra.net.Outbo
>> undTcpConnection.run(OutboundTcpConnection.java:262)
>> [apache-cassandra-3.11.0.jar:3.11.0]
>>
>> ```
>>
>>
>>
>> Basically, it’s tons and tons of the same message over and over (on all
>> clusters, all C* nodes). It tries roughly 4-5 times a second to open a tcp
>> connection to localhost (?) for gossiping.
>>
>>
>>
>> What we know:
>>
>> - does not happen on Cassandra 3.0.15, but happen on 3.11.1 (same
>> configuration).
>>
>> - does happen even on minikube-single-Cassandra “cluster”.
>>
>> - does not happen on docker-compose Cassandra cluster, only on kubernetes
>> one.
>>
>>
>>
>> Our configuration is pretty much this helm chart:
>> https://github.com/kubernetes/charts/blob/master/incubator/c
>> assandra/values.yaml
>>
>>
>>
>> Do you have any idea what it could be related to?
>>
>>
>>
>
>

Mime
View raw message