zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From εΎζœζ™– <xuzhaohui1...@163.com>
Subject client cannot connect to zookeeper after node replacement
Date Fri, 11 May 2018 12:13:55 GMT
Hi there,



We were recently running into an issue in cloud env.
Specifically, we have a kafka cluster setup in cloud, and it turns out to be a failure when
the zookeeper node is replaced, namely the ip changed while the hostname remains same. We
have patched PR150 (https://github.com/apache/zookeeper/pull/150/files) but it seems still
not works. While the new zk node ip can be re-resolved and zk client in kafka can establish
connection to zk node, but the client seems not able to read content from the connection.
Anyone encounters similar issue?

Pasted the log from kafka server.

[2018-05-11 03:47:46,806] WARN SASL configuration failed: javax.security.auth.login.LoginException:
No JAAS configuration section named 'Client' was found in specified JAAS configuration file:
'/usr/share/kafka/config/kafka_server_jaas.
conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper
server allows it. (org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:46,806] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-05-11 03:47:46,806] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-05-11 03:47:46,806] INFO Opening socket connection to server host.name/10.147.164.83:2181
(org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:49,815] WARN Session 0x1634d42d88a001e for server null, unexpected error,
closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
[2018-05-11 03:47:50,667] INFO Resolving again hostname: host.name (org.apache.zookeeper.client.StaticHostProvider)
[2018-05-11 03:47:50,667] WARN Cannot re-resolve server: host.name/10.147.164.83:2181 (org.apache.zookeeper.client.StaticHostProvider)
java.net.UnknownHostException: host.name
at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:142)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
[2018-05-11 03:47:51,668] WARN SASL configuration failed: javax.security.auth.login.LoginException:
No JAAS configuration section named 'Client' was found in specified JAAS configuration file:
'/usr/share/kafka/config/kafka_server_jaas.
conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper
server allows it. (org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:51,668] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-05-11 03:47:51,668] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-05-11 03:47:51,668] INFO Opening socket connection to server host.name/10.147.164.83:2181
(org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:52,822] WARN Session 0x1634d42d88a001e for server null, unexpected error,
closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
[2018-05-11 03:47:53,427] INFO Resolving again hostname: host.name (org.apache.zookeeper.client.StaticHostProvider)
[2018-05-11 03:47:54,429] WARN SASL configuration failed: javax.security.auth.login.LoginException:
No JAAS configuration section named 'Client' was found in specified JAAS configuration file:
'/usr/share/kafka/config/kafka_server_jaas.
conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper
server allows it. (org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:54,429] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-05-11 03:47:54,429] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-05-11 03:47:54,429] INFO Opening socket connection to server host.name/10.147.161.203:2181
(org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:55,383] INFO Socket connection established to host.name/10.147.161.203:2181,
initiating session (org.apache.zookeeper.ClientCnxn)
[2018-05-11 03:47:55,386] INFO Unable to read additional data from server sessionid 0x1634d42d88a001e,
likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)




this ends up with continuous log flushing, the log is repeated in a sequence like above highlighted
ones.




Anyone can share thoughts regarding this issue?




Thanks,

Jeff





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message