hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@cloudera.com>
Subject Re: python client structure
Date Wed, 21 Apr 2010 07:26:03 GMT
Hi Travis -

Great to see zkpython getting used. I'm glad you're finding the problems
with the documentation - please do file JIRAs with anything you'd like to
see improved (and I know there's a lot to improve with zkpython).

You are using the asynchronous form of get_children. This means that
ZooKeeper can send you two notifications. The first is called when the
get_children call completes. The second is the watcher and is called when
the children of the watched node change. You can omit the watcher if you
don't need it, or alternatively use the synchronous form which is written
get_children. This call doesn't return until the operation is complete, so
you don't need to worry about a callback.

The first argument to any watcher or callback is the handle of the client
that placed the callback. Not the return code! We pass that in so that it's
easy to make further ZK calls because the handle is readily available. The
second argument for a callback is the return code, and that  can be mapped
to a string via zerror(rc) if needed (but as you have found, there are
numeric return code constants in the module that have readable symbolic
names).

Does this help at all? Let me know if you have any follow on questions.

cheers,
Henry

On 20 April 2010 23:33, Travis Crawford <traviscrawford@gmail.com> wrote:

> Hey zookeeper gurus -
>
> I'm getting started with Zookeeper and the python client and an curious if
> I'm structuring watches correctly. I'd like to watch a znode and do stuff
> when its children change. Something doesn't feel right about having two
> methods: one to handle the actual get children call, and one to handle the
> watch.
>
> Does this seem like the right direction? If not, any suggestions on how to
> better structure things?
>
>
> #!/usr/bin/python
>
> import signal
> import threading
> import zookeeper
>
> import logging
> logger = logging.getLogger()
>
> from optparse import OptionParser
> options = None
> args = None
>
>
> class ZKTest(threading.Thread):
>  zparent = '/home/travis/zktest'
>
>  def __init__(self):
>    threading.Thread.__init__(self)
>    if options.verbose:
>      zookeeper.set_debug_level(zookeeper.LOG_LEVEL_DEBUG)
>    self.zh = zookeeper.init(options.servers)
>    zookeeper.aget_children(self.zh, self.zparent, self.watcher,
> self.handler)
>
>  def __del__(self):
>    zookeeper.close(self.zh)
>
>  def handler(self, rc, rc1, children):
>    """Handle zookeeper.aget_children() responses.
>
>    Args:
>      Arguments are not documented well and I'm not entirely sure what to
>      call these. ``rc`` appears to be the response code, such as OK.
>      However, the only possible mapping of 0 is OK, so in successful cases
>      there appear to be two response codes. The example with no children
>      returned ``rc1`` of -7 which maps to OPERATIONTIMEOUT so that appears
>      to be an error code, but its not clear what was OK in that case.
>
>      If anyone figures this out I would love to know.
>
>    Example args:
>      'args': (0, 0, ['a', 'b'])
>      'args': (0, -7, [])
>
>    Does not provide a return value.
>    """
>    logger.debug('Processing response: (%d, %d, %s)' % (rc, rc1, children))
>    if (zookeeper.OK == rc and zookeeper.OK == rc1):
>      logger.debug('Do the actual work here.')
>    else:
>      logger.debug('Error getting children! Retrying.')
>      zookeeper.aget_children(self.zh, self.zparent, self.watcher,
> self.handler)
>
>  def watcher(self, rc, event, state, path):
>    """Handle zookeeper.aget_children() watches.
>
>    This code is called when an child znode changes and triggers a child
>    watch. It is not called to handle the aget_children call itself.
>
>    Numeric arguments map to constants. See ``DATA`` in ``help(zookeeper)``
>    for more information.
>
>    Args:
>      rc Return code.
>      event Event that caused the watch (often called ``type`` elsewhere).
>      stats Connection state.
>      path Znode that triggered this watch.
>
>    Does not provide a return value.
>    """
>    logger.debug('Child watch: (%d, %d, %d, %s)' % (rc, event, state, path))
>    zookeeper.aget_children(self.zh, self.zparent, self.watcher,
> self.handler)
>
>  def run(self):
>    while True:
>      pass
>
>
> def main():
>  # Allow Ctrl-C
>  signal.signal(signal.SIGINT, signal.SIG_DFL)
>
>  parser = OptionParser()
>  parser.add_option('-v', '--verbose',
>    dest='verbose',
>    default=True,
>    action='store_true',
>    help='Verbose logging. (default: %default)')
>  parser.add_option('--servers',
>    dest='servers',
>    default='localhost:2181',
>    help='Comma-separated list of host:port pairs. (default: %default)')
>  global options
>  global args
>  (options, args) = parser.parse_args()
>
>  if options.verbose:
>    logger.setLevel(logging.DEBUG)
>  else:
>    logger.setLevel(logging.INFO)
>  formatter = logging.Formatter("%(asctime)s %(filename)s:%(lineno)d -
> %(message)s")
>  stream_handler = logging.StreamHandler()
>  stream_handler.setFormatter(formatter)
>  logger.addHandler(stream_handler)
>
>  zktest = ZKTest()
>  zktest.daemon = True
>  zktest.start()
>
>
> if __name__ == '__main__':
>  main()
>
>
> Thanks!
> Travis
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message