lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Lerner <joeler...@gmail.com>
Subject Continuous Zookeeper Client Warnings
Date Fri, 04 Jan 2019 12:59:29 GMT
Hi, 

We have a simple architecture: 2 SOLR Cloud servers (on servers #1 and #2), 
and 3 zookeeper instances (on servers #1, #2, and #3). Things appear to work 
fine but: 

We are getting *TONS* of continuous log warnings from our client 
applications. From one server it shows this: 


[MYAPP-WEB] 2019-01-03 14:17:46,519 WARN
[org.apache.solr.common.cloud.ConnectionManager] - <Watcher
org.apache.solr.common.cloud.ConnectionManager@a38d23a name:
ZooKeeperConnection
Watcher:888.88.101.65:2181,888.88.101.64:2181,888.88.101.63:2181 got event
WatchedEvent state:Disconnected type:None path:null path: null type: None>
[MYAPP-WEB] 2019-01-03 14:17:46,519 WARN
[org.apache.solr.common.cloud.ConnectionManager] - <zkClient has
disconnected>
[MYAPP-WEB] 2019-01-03 14:17:47,385 INFO [org.apache.zookeeper.ClientCnxn] -
<Opening socket connection to server 888.88.101.64/888.88.101.64:2181. Will
not attempt to authenticate using SASL (unknown error)>
[MYAPP-WEB] 2019-01-03 14:17:47,386 INFO [org.apache.zookeeper.ClientCnxn] -
<Socket connection established to 888.88.101.64/888.88.101.64:2181,
initiating session>
[MYAPP-WEB] 2019-01-03 14:17:47,386 INFO [org.apache.zookeeper.ClientCnxn] -
<Session establishment complete on server 888.88.101.64/888.88.101.64:2181,
sessionid = 0x2679a04fc7b0004, negotiated timeout = 10000>
[MYAPP-WEB] 2019-01-03 14:17:47,386 INFO
[org.apache.solr.common.cloud.ConnectionManager] - <zkClient has connected>
[MYAPP-WEB] 2019-01-03 14:17:47,386 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x2679a04fc7b0004 for server 888.88.101.64/888.88.101.64:2181,
unexpected error, closing socket connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/proto/WatcherEvent 
        at
org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:770) 
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:94) 
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144) 
[MYAPP-WEB] 2019-01-03 14:17:47,487 WARN
[org.apache.solr.common.cloud.ConnectionManager] - <Watcher
org.apache.solr.common.cloud.ConnectionManager@a38d23a name:
ZooKeeperConnection
Watcher:888.88.101.65:2181,888.88.101.64:2181,888.88.101.63:2181 got event
WatchedEvent state:Disconnected type:None path:null path: null type: None>
[MYAPP-WEB] 2019-01-03 14:17:47,487 WARN
[org.apache.solr.common.cloud.ConnectionManager] - <zkClient has
disconnected>
[MYAPP-WEB] 2019-01-03 14:17:47,943 INFO [org.apache.zookeeper.ClientCnxn] -
<Opening socket connection to server 888.88.101.63/888.88.101.63:2181. Will
not attempt to authenticate using SASL (unknown error)>
[MYAPP-WEB] 2019-01-03 14:17:47,943 INFO [org.apache.zookeeper.ClientCnxn] -
<Socket connection established to 888.88.101.63/888.88.101.63:2181,
initiating session>
[MYAPP-WEB] 2019-01-03 14:17:47,943 INFO [org.apache.zookeeper.ClientCnxn] -
<Session establishment complete on server 888.88.101.63/888.88.101.63:2181,
sessionid = 0x2679a04fc7b0004, negotiated timeout = 10000>
[MYAPP-WEB] 2019-01-03 14:17:47,944 INFO
[org.apache.solr.common.cloud.ConnectionManager] - <zkClient has connected>
[MYAPP-WEB] 2019-01-03 14:17:47,944 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x2679a04fc7b0004 for server 888.88.101.63/888.88.101.63:2181,
unexpected error, closing socket connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/proto/WatcherEvent 
        at
org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:770) 
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:94) 
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144) 
[MYAPP-WEB] 2019-01-03 14:17:48,044 WARN
[org.apache.solr.common.cloud.ConnectionManager] - <Watcher
org.apache.solr.common.cloud.ConnectionManager@a38d23a name:
ZooKeeperConnection
Watcher:888.88.101.65:2181,888.88.101.64:2181,888.88.101.63:2181 got event
WatchedEvent state:Disconnected type:None path:null path: null type: None>
[MYAPP-WEB] 2019-01-03 14:17:48,044 WARN
[org.apache.solr.common.cloud.ConnectionManager] - <zkClient has
disconnected>
[MYAPP-WEB] 2019-01-03 14:17:48,687 INFO [org.apache.zookeeper.ClientCnxn] -
<Opening socket connection to server 888.88.101.65/888.88.101.65:2181. Will
not attempt to authenticate using SASL (unknown error)>
[MYAPP-WEB] 2019-01-03 14:17:48,687 INFO [org.apache.zookeeper.ClientCnxn] -
<Socket connection established to 888.88.101.65/888.88.101.65:2181,
initiating session>
[MYAPP-WEB] 2019-01-03 14:17:48,688 INFO [org.apache.zookeeper.ClientCnxn] -
<Session establishment complete on server 888.88.101.65/888.88.101.65:2181,
sessionid = 0x2679a04fc7b0004, negotiated timeout = 10000>
[MYAPP-WEB] 2019-01-03 14:17:48,689 INFO
[org.apache.solr.common.cloud.ConnectionManager] - <zkClient has connected>
[MYAPP-WEB] 2019-01-03 14:17:48,689 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x2679a04fc7b0004 for server 888.88.101.65/888.88.101.65:2181,
unexpected error, closing socket connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/proto/WatcherEvent 
        at
org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:770) 
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:94) 
        at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144) 






And from another server we get this: 


[MYAPP-WEB] 2019-01-03 14:19:47,273 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x3679a09efbb0002 for server null, unexpected error, closing socket
connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/Login 
        at
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:216)

        at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:119)

        at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1011) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 
[MYAPP-WEB] 2019-01-03 14:19:47,753 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x3679a09efbb0002 for server null, unexpected error, closing socket
connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/Login 
        at
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:216)

        at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:119)

        at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1011) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 
[MYAPP-WEB] 2019-01-03 14:19:48,197 INFO
[gov.fbi.guardian.web.filter.SentinelRedirectFilter] - <Sentinel Redirect
touched>
[MYAPP-WEB] 2019-01-03 14:19:48,450 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x3679a09efbb0002 for server null, unexpected error, closing socket
connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/Login 
        at
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:216)

        at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:119)

        at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1011) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 
[MYAPP-WEB] 2019-01-03 14:19:49,912 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x3679a09efbb0002 for server null, unexpected error, closing socket
connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/Login 
        at
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:216)

        at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:119)

        at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1011) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 
[MYAPP-WEB] 2019-01-03 14:19:50,977 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x3679a09efbb0002 for server null, unexpected error, closing socket
connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/Login 
        at
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:216)

        at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:119)

        at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1011) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 
[MYAPP-WEB] 2019-01-03 14:19:51,233 WARN [org.apache.zookeeper.ClientCnxn] -
<Session 0x3679a09efbb0002 for server null, unexpected error, closing socket
connection and attempting reconnect>
java.lang.NoClassDefFoundError: org/apache/zookeeper/Login 
        at
org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslClient(ZooKeeperSaslClient.java:216)

        at
org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:119)

        at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1011) 
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 





These are making our application logs impossible to read, and I assume
indicate 
that something is wrong. 

Thanks for any help! 

Joe Lerner 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Mime
View raw message