I'm running a Tornado webserver and using ZooKeeper to store some metadata and occasionally
the ZooKeeper connection will error out irrevocably. Any subsequent calls to ZooKeeper from
this process will result in a SystemError.
Here is the relevant portion of the Python traceback:
<snip>...
File "/usr/lib/pymodules/python2.5/zuul/storage/zoo.py", line 69, in call
return getattr(zookeeper, name)(self.handle, *args)
SystemError: NULL result without error in PyObject_Call
I found this in the ZooKeeper server logs:
2010-07-13 06:52:46,488 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251]
- Accepted socket connection from /10.2.128.233:54779
2010-07-13 06:52:46,489 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@742]
- Client attempting to renew session 0x429b865a6270003 at /10.2.128.233:54779
2010-07-13 06:52:46,489 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:Learner@95] - Revalidating
client: 299973596915630083
2010-07-13 06:52:46,793 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:NIOServerCnxn@1424] - Invalid
session 0x429b865a6270003 for client /10.2.128.233:54779, probably expired
2010-07-13 06:52:46,794 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1286]
- Closed socket connection for client /10.2.128.233:54779 which had sessionid 0x429b865a6270003
The ZooKeeper ensemble is healthy; each node responds as expected to the four letter word
commands and a simple restart of the Tornado processes "fixes" this.
My question is, if this really is due to session expiration why is a SessionExpiredException
not raised? Another question, is there an easy way to determine the version of the ZooKeeper
Python bindings I'm using? I built the 3.3.0 bindings but I just want to be able to verify
that.
Thanks for the help,
Rich
|