hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chang Song (JIRA)" <j...@apache.org>
Subject [jira] Created: (ZOOKEEPER-919) Ephemeral nodes remains in one of ensemble after deliberate SIGKILL
Date Thu, 04 Nov 2010 13:43:44 GMT
Ephemeral nodes remains in one of ensemble after deliberate SIGKILL
-------------------------------------------------------------------

                 Key: ZOOKEEPER-919
                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-919
             Project: Zookeeper
          Issue Type: Bug
          Components: server
    Affects Versions: 3.3.1
         Environment: Linux CentOS 5.3 64bit, JDK 1.6.0-22
            Reporter: Chang Song


I was testing stability of Zookeeper ensemble for production deployment. Three node ensemble
cluster configuration.
In a loop, I kill/restart three Zookeeper clients that created one ephemeral node each, and
at the same time,
I killed Java process on one of ensemble (dont' know if it was a leader or not). Then I restarted
Zookeeper on the server,

It turns out that on two zookeeper ensemble servers, all the ephemeral nodes are gone (it
should), but on the newly started
Zookeeper server, the two old ephemeral nodes stayed.  The zookeeper didn't restart in standalone
mode since new ephemeral
nodes gets created on all ensemble servers. 
I captured the log.


2010-11-04 17:48:50,201 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17288:NIOServerCnxn$Factory@250]
- Accepted socket connection from /10.25.131.21:11191
2010-11-04 17:48:50,202 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17288:NIOServerCnxn@776]
- Client attempting to establish new session at /10.25.131.21:11191
2010-11-04 17:48:50,203 - INFO  [CommitProcessor:1:NIOServerCnxn@1579] - Established session
0x12c160c31fc000b with negotiated timeout 30000 for client /10.25.131.21:11191
2010-11-04 17:48:50,206 - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17288:NIOServerCnxn@633]
- EndOfStreamException: Unable to read additional data from client sessionid 0x12c160c31fc000b,
likely client has closed socket
2010-11-04 17:48:50,207 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:17288:NIOServerCnxn@1434]
- Closed socket connection for client /10.25.131.21:11191 which had sessionid 0x12c160c31fc000b
2010-11-04 17:48:50,207 - ERROR [CommitProcessor:1:NIOServerCnxn@444] - Unexpected Exception:
java.nio.channels.CancelledKeyException
        at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
        at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
        at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:417)
        at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1508)
        at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:367)
        at org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:73)


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message