curator-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhaogang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CURATOR-165) LeaderLatch.checkLeadership(LeaderLatch.java:478) will throw a exception, then no leader will be elected.
Date Thu, 13 Nov 2014 02:04:16 GMT

    [ https://issues.apache.org/jira/browse/CURATOR-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209093#comment-14209093
] 

zhaogang commented on CURATOR-165:
----------------------------------

This error occured at the time that the when the client lost connection and reconnected to
the zookeeper again, here is detail logs:

sing socket connection and attempting reconnect
INFO   | jvm 1    | 2014/10/30 00:24:38 | 30 Oct 2014 00:24:38 INFO  org.apache.curator.framework.state.ConnectionStateManager
- State change: SUSPENDED
INFO   | jvm 1    | 2014/10/30 00:24:38 | 30 Oct 2014 00:24:38 INFO  org.apache.curator.framework.state.ConnectionStateManager
- State change: SUSPENDED
INFO   | jvm 1    | 2014/10/30 00:24:39 | 30 Oct 2014 00:24:39 INFO  org.apache.zookeeper.ClientCnxn
- Opening socket connection to server 10.67.90.141/10.67.90.141:2181. Will not attempt to
authenticate using SASL (unknown error)
INFO   | jvm 1    | 2014/10/30 00:24:39 | 30 Oct 2014 00:24:39 WARN  org.apache.zookeeper.ClientCnxn
- Session 0x1489e3e9cce0011 for server null, unexpected error, closing socket connection and
attempting reconnect
INFO   | jvm 1    | 2014/10/30 00:24:39 | java.net.ConnectException: Connection refused
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
INFO   | jvm 1    | 2014/10/30 00:24:39 | 30 Oct 2014 00:24:39 INFO  org.apache.zookeeper.ClientCnxn
- Opening socket connection to server 10.67.90.141/10.67.90.141:2181. Will not attempt to
authenticate using SASL (unknown error)
INFO   | jvm 1    | 2014/10/30 00:24:39 | 30 Oct 2014 00:24:39 WARN  org.apache.zookeeper.ClientCnxn
- Session 0x2487088d7080011 for server null, unexpected error, closing socket connection and
attempting reconnect
INFO   | jvm 1    | 2014/10/30 00:24:39 | java.net.ConnectException: Connection refused
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
INFO   | jvm 1    | 2014/10/30 00:24:39 |       at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.zookeeper.ClientCnxn
- Opening socket connection to server 10.67.90.142/10.67.90.142:2181. Will not attempt to
authenticate using SASL (unknown error)
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.zookeeper.ClientCnxn
- Socket connection established to 10.67.90.142/10.67.90.142:2181, initiating session
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.zookeeper.ClientCnxn
- Session establishment complete on server 10.67.90.142/10.67.90.142:2181, sessionid = 0x1489e3e9cce0011,
negotiated timeout = 40000
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.curator.framework.state.ConnectionStateManager
- State change: RECONNECTED
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.zookeeper.ClientCnxn
- Opening socket connection to server 10.67.90.142/10.67.90.142:2181. Will not attempt to
authenticate using SASL (unknown error)
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.zookeeper.ClientCnxn
- Socket connection established to 10.67.90.142/10.67.90.142:2181, initiating session
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.zookeeper.ClientCnxn
- Session establishment complete on server 10.67.90.142/10.67.90.142:2181, sessionid = 0x2487088d7080011,
negotiated timeout = 40000
INFO   | jvm 1    | 2014/10/30 00:24:40 | 30 Oct 2014 00:24:40 INFO  org.apache.curator.framework.state.ConnectionStateManager
- State change: RECONNECTED
DEBUG  | wrapperp | 2014/10/30 00:24:42 | send a packet PING : ping 00fc2c4f
INFO   | jvm 1    | 2014/10/30 00:24:42 | WrapperManager Debug: Received a packet PING : ping
00fc2c4f
INFO   | jvm 1    | 2014/10/30 00:24:42 | WrapperManager Debug: Send a packet PING : ping
00fc2c4f
DEBUG  | wrapperp | 2014/10/30 00:24:42 | read a packet PING : ping 00fc2c4f
DEBUG  | wrapperp | 2014/10/30 00:24:46 | send a packet PING : ping 00fc2c78
INFO   | jvm 1    | 2014/10/30 00:24:46 | WrapperManager Debug: Received a packet PING : ping
00fc2c78
INFO   | jvm 1    | 2014/10/30 00:24:46 | WrapperManager Debug: Send a packet PING : ping
00fc2c78
DEBUG  | wrapperp | 2014/10/30 00:24:46 | read a packet PING : ping 00fc2c78
INFO   | jvm 1    | 2014/10/30 00:24:49 | 30 Oct 2014 00:24:49 ERROR org.apache.curator.framework.imps.CuratorFrameworkImpl
- Background exception was not retry-able or retry gave up
INFO   | jvm 1    | 2014/10/30 00:24:49 | java.lang.IllegalArgumentException: Path must start
with / character
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:53)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.utils.ZKPaths.getNodeFromPath(ZKPaths.java:56)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.framework.recipes.leader.LeaderLatch.checkLeadership(LeaderLatch.java:478)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.framework.recipes.leader.LeaderLatch.access$500(LeaderLatch.java:60)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.framework.recipes.leader.LeaderLatch$6.processResult(LeaderLatch.java:536)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:730)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:516)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
INFO   | jvm 1    | 2014/10/30 00:24:49 |       at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
DEBUG  | wrapperp | 2014/10/30 00:24:50 | send a packet PING : ping 00fc2ca1
INFO   | jvm 1    | 2014/10/30 00:24:50 | WrapperManager Debug: Received a packet PING : ping
00fc2ca1
INFO   | jvm 1    | 2014/10/30 00:24:50 | WrapperManager Debug: Send a packet PING : ping
00fc2ca1
DEBUG  | wrapperp | 2014/10/30 00:24:50 | read a packet PING : ping 00fc2ca1
DEBUG  | wrapperp | 2014/10/30 00:24:54 | send a packet PING : ping 00fc2cca
INFO   | jvm 1    | 2014/10/30 00:24:54 | WrapperManager Debug: Received a packet PING : ping
00fc2cca
INFO   | jvm 1    | 2014/10/30 00:24:54 | WrapperManager Debug: Send a packet PING : ping
00fc2cca
DEBUG  | wrapperp | 2014/10/30 00:24:54 | read a packet PING : ping 00fc2cca
DEBUG  | wrapperp | 2014/10/30 00:24:58 | send a packet PING : ping 00fc2cf3
INFO   | jvm 1    | 2014/10/30 00:24:59 | WrapperManager Debug: Received a packet PING : ping
00fc2cf3
INFO   | jvm 1    | 2014/10/30 00:24:59 | WrapperManager Debug: Send a packet PING : ping
00fc2cf3
DEBUG  | wrapperp | 2014/10/30 00:24:59 | read a packet PING : ping 00fc2cf3
DEBUG  | wrapperp | 2014/10/30 00:25:03 | send a packet PING : ping 00fc2d1c
INFO   | jvm 1    | 2014/10/30 00:25:03 | WrapperManager Debug: Received a packet PING : ping
00fc2d1c
INFO   | jvm 1    | 2014/10/30 00:25:03 | WrapperManager Debug: Send a packet PING : ping
00fc2d1c
DEBUG  | wrapperp | 2014/10/30 00:25:03 | read a packet PING : ping 00fc2d1c
DEBUG  | wrapperp | 2014/10/30 00:25:07 | send a packet PING : ping 00fc2d45
INFO   | jvm 1    | 2014/10/30 00:25:07 | WrapperManager Debug: Received a packet PING : ping
00fc2d45
INFO   | jvm 1    | 2014/10/30 00:25:07 | WrapperManager Debug: Send a packet PING : ping
00fc2d45


> LeaderLatch.checkLeadership(LeaderLatch.java:478) will throw a exception, then no leader
will be elected.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: CURATOR-165
>                 URL: https://issues.apache.org/jira/browse/CURATOR-165
>             Project: Apache Curator
>          Issue Type: Bug
>          Components: Recipes
>    Affects Versions: 2.5.0
>            Reporter: zhaogang
>            Priority: Critical
>
> We use curator for leader election within our 2 clients, sometimes none of the 2 clients
was a leader.
> We checked the log and found this caused by a exception which was being throwed in method
LeaderLatch.checkLeadership(), after throwing this exception the curator won't do the leader
election. Here is error log:
> ERROR org.apache.curator.framework.imps.CuratorFrameworkImpl - Background exception was
not retry-able or retry gave up
> INFO   | jvm 1    | 2014/11/12 00:36:52 | java.lang.IllegalArgumentException: Path must
start with / character
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:53)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.utils.ZKPaths.getNodeFromPath(ZKPaths.java:56)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.framework.recipes.leader.LeaderLatch.checkLeadership(LeaderLatch.java:478)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.framework.recipes.leader.LeaderLatch.access$500(LeaderLatch.java:60)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.framework.recipes.leader.LeaderLatch$6.processResult(LeaderLatch.java:536)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:730)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:516)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> INFO   | jvm 1    | 2014/11/12 00:36:52 | 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message