zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andor Molnar <an...@cloudera.com>
Subject Re: client cannot connect to zookeeper after node replacement
Date Fri, 11 May 2018 16:01:47 GMT
Hi Jeff,

There's another patch which is supposed to be the successor of #151 here:

https://github.com/apache/zookeeper/pull/451

Would you please try this one too and make sure it fixes the problem?

Thanks,
Andor




On Fri, May 11, 2018 at 5:13 AM, εΎζœζ™– <xuzhaohui1988@163.com> wrote:

> Hi there,
>
>
>
> We were recently running into an issue in cloud env.
> Specifically, we have a kafka cluster setup in cloud, and it turns out to
> be a failure when the zookeeper node is replaced, namely the ip changed
> while the hostname remains same. We have patched PR150 (
> https://github.com/apache/zookeeper/pull/150/files) but it seems still
> not works. While the new zk node ip can be re-resolved and zk client in
> kafka can establish connection to zk node, but the client seems not able to
> read content from the connection. Anyone encounters similar issue?
>
> Pasted the log from kafka server.
>
> [2018-05-11 03:47:46,806] WARN SASL configuration failed:
> javax.security.auth.login.LoginException: No JAAS configuration section
> named 'Client' was found in specified JAAS configuration file:
> '/usr/share/kafka/config/kafka_server_jaas.
> conf'. Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it. (org.apache.zookeeper.
> ClientCnxn)
> [2018-05-11 03:47:46,806] ERROR [ZooKeeperClient] Auth failed.
> (kafka.zookeeper.ZooKeeperClient)
> [2018-05-11 03:47:46,806] ERROR [ZooKeeperClient] Auth failed.
> (kafka.zookeeper.ZooKeeperClient)
> [2018-05-11 03:47:46,806] INFO Opening socket connection to server
> host.name/10.147.164.83:2181 (org.apache.zookeeper.ClientCnxn)
> [2018-05-11 03:47:49,815] WARN Session 0x1634d42d88a001e for server null,
> unexpected error, closing socket connection and attempting reconnect
> (org.apache.zookeeper.ClientCnxn)
> java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
> ClientCnxnSocketNIO.java:361)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
> [2018-05-11 03:47:50,667] INFO Resolving again hostname: host.name
> (org.apache.zookeeper.client.StaticHostProvider)
> [2018-05-11 03:47:50,667] WARN Cannot re-resolve server:
> host.name/10.147.164.83:2181 (org.apache.zookeeper.client.
> StaticHostProvider)
> java.net.UnknownHostException: host.name
> at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
> at java.net.InetAddress.getAllByName(InetAddress.java:1192)
> at java.net.InetAddress.getAllByName(InetAddress.java:1126)
> at org.apache.zookeeper.client.StaticHostProvider.next(
> StaticHostProvider.java:142)
> at org.apache.zookeeper.ClientCnxn$SendThread.
> startConnect(ClientCnxn.java:997)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
> [2018-05-11 03:47:51,668] WARN SASL configuration failed:
> javax.security.auth.login.LoginException: No JAAS configuration section
> named 'Client' was found in specified JAAS configuration file:
> '/usr/share/kafka/config/kafka_server_jaas.
> conf'. Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it. (org.apache.zookeeper.
> ClientCnxn)
> [2018-05-11 03:47:51,668] ERROR [ZooKeeperClient] Auth failed.
> (kafka.zookeeper.ZooKeeperClient)
> [2018-05-11 03:47:51,668] ERROR [ZooKeeperClient] Auth failed.
> (kafka.zookeeper.ZooKeeperClient)
> [2018-05-11 03:47:51,668] INFO Opening socket connection to server
> host.name/10.147.164.83:2181 (org.apache.zookeeper.ClientCnxn)
> [2018-05-11 03:47:52,822] WARN Session 0x1634d42d88a001e for server null,
> unexpected error, closing socket connection and attempting reconnect
> (org.apache.zookeeper.ClientCnxn)
> java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(
> ClientCnxnSocketNIO.java:361)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
> [2018-05-11 03:47:53,427] INFO Resolving again hostname: host.name
> (org.apache.zookeeper.client.StaticHostProvider)
> [2018-05-11 03:47:54,429] WARN SASL configuration failed:
> javax.security.auth.login.LoginException: No JAAS configuration section
> named 'Client' was found in specified JAAS configuration file:
> '/usr/share/kafka/config/kafka_server_jaas.
> conf'. Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it. (org.apache.zookeeper.
> ClientCnxn)
> [2018-05-11 03:47:54,429] ERROR [ZooKeeperClient] Auth failed.
> (kafka.zookeeper.ZooKeeperClient)
> [2018-05-11 03:47:54,429] ERROR [ZooKeeperClient] Auth failed.
> (kafka.zookeeper.ZooKeeperClient)
> [2018-05-11 03:47:54,429] INFO Opening socket connection to server
> host.name/10.147.161.203:2181 (org.apache.zookeeper.ClientCnxn)
> [2018-05-11 03:47:55,383] INFO Socket connection established to
> host.name/10.147.161.203:2181, initiating session (org.apache.zookeeper.
> ClientCnxn)
> [2018-05-11 03:47:55,386] INFO Unable to read additional data from server
> sessionid 0x1634d42d88a001e, likely server has closed socket, closing
> socket connection and attempting reconnect (org.apache.zookeeper.
> ClientCnxn)
>
>
>
>
> this ends up with continuous log flushing, the log is repeated in a
> sequence like above highlighted ones.
>
>
>
>
> Anyone can share thoughts regarding this issue?
>
>
>
>
> Thanks,
>
> Jeff
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message