hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Travis Crawford <traviscrawf...@gmail.com>
Subject Re: python client structure
Date Wed, 21 Apr 2010 17:03:40 GMT
On Wed, Apr 21, 2010 at 12:26 AM, Henry Robinson <henry@cloudera.com> wrote:
>
> Hi Travis -
>
> Great to see zkpython getting used. I'm glad you're finding the problems
> with the documentation - please do file JIRAs with anything you'd like to
> see improved (and I know there's a lot to improve with zkpython).
>

Yeah I was pretty excited to see the python bindings. Thanks!

The best place for help I was able to find is ``help(zookeeper)`` --
is there somewhere else I should be looking instead? It looks like you
wrapped the c client, so I also have been looking in zookeeper.h and
trying to infer what's going on.


> You are using the asynchronous form of get_children. This means that
> ZooKeeper can send you two notifications. The first is called when the
> get_children call completes. The second is the watcher and is called when
> the children of the watched node change. You can omit the watcher if you
> don't need it, or alternatively use the synchronous form which is written
> get_children. This call doesn't return until the operation is complete, so
> you don't need to worry about a callback.
>

Ok sounds like that general structure will work then. Thanks for verifying.


> The first argument to any watcher or callback is the handle of the client
> that placed the callback. Not the return code! We pass that in so that it's
> easy to make further ZK calls because the handle is readily available. The
> second argument for a callback is the return code, and that  can be mapped
> to a string via zerror(rc) if needed (but as you have found, there are
> numeric return code constants in the module that have readable symbolic
> names).
>

Aah first argument is the handle! That makes sense.


> Does this help at all? Let me know if you have any follow on questions.

Very much so! If I cleanup the sample code below would you want to put
this on a wiki or check in as an example? Would have been nice if
someone already figured this out when I started messing with things :)

--travis


>
> cheers,
> Henry
>
> On 20 April 2010 23:33, Travis Crawford <traviscrawford@gmail.com> wrote:
>
> > Hey zookeeper gurus -
> >
> > I'm getting started with Zookeeper and the python client and an curious if
> > I'm structuring watches correctly. I'd like to watch a znode and do stuff
> > when its children change. Something doesn't feel right about having two
> > methods: one to handle the actual get children call, and one to handle the
> > watch.
> >
> > Does this seem like the right direction? If not, any suggestions on how to
> > better structure things?
> >
> >
> > #!/usr/bin/python
> >
> > import signal
> > import threading
> > import zookeeper
> >
> > import logging
> > logger = logging.getLogger()
> >
> > from optparse import OptionParser
> > options = None
> > args = None
> >
> >
> > class ZKTest(threading.Thread):
> >  zparent = '/home/travis/zktest'
> >
> >  def __init__(self):
> >    threading.Thread.__init__(self)
> >    if options.verbose:
> >      zookeeper.set_debug_level(zookeeper.LOG_LEVEL_DEBUG)
> >    self.zh = zookeeper.init(options.servers)
> >    zookeeper.aget_children(self.zh, self.zparent, self.watcher,
> > self.handler)
> >
> >  def __del__(self):
> >    zookeeper.close(self.zh)
> >
> >  def handler(self, rc, rc1, children):
> >    """Handle zookeeper.aget_children() responses.
> >
> >    Args:
> >      Arguments are not documented well and I'm not entirely sure what to
> >      call these. ``rc`` appears to be the response code, such as OK.
> >      However, the only possible mapping of 0 is OK, so in successful cases
> >      there appear to be two response codes. The example with no children
> >      returned ``rc1`` of -7 which maps to OPERATIONTIMEOUT so that appears
> >      to be an error code, but its not clear what was OK in that case.
> >
> >      If anyone figures this out I would love to know.
> >
> >    Example args:
> >      'args': (0, 0, ['a', 'b'])
> >      'args': (0, -7, [])
> >
> >    Does not provide a return value.
> >    """
> >    logger.debug('Processing response: (%d, %d, %s)' % (rc, rc1, children))
> >    if (zookeeper.OK == rc and zookeeper.OK == rc1):
> >      logger.debug('Do the actual work here.')
> >    else:
> >      logger.debug('Error getting children! Retrying.')
> >      zookeeper.aget_children(self.zh, self.zparent, self.watcher,
> > self.handler)
> >
> >  def watcher(self, rc, event, state, path):
> >    """Handle zookeeper.aget_children() watches.
> >
> >    This code is called when an child znode changes and triggers a child
> >    watch. It is not called to handle the aget_children call itself.
> >
> >    Numeric arguments map to constants. See ``DATA`` in ``help(zookeeper)``
> >    for more information.
> >
> >    Args:
> >      rc Return code.
> >      event Event that caused the watch (often called ``type`` elsewhere).
> >      stats Connection state.
> >      path Znode that triggered this watch.
> >
> >    Does not provide a return value.
> >    """
> >    logger.debug('Child watch: (%d, %d, %d, %s)' % (rc, event, state, path))
> >    zookeeper.aget_children(self.zh, self.zparent, self.watcher,
> > self.handler)
> >
> >  def run(self):
> >    while True:
> >      pass
> >
> >
> > def main():
> >  # Allow Ctrl-C
> >  signal.signal(signal.SIGINT, signal.SIG_DFL)
> >
> >  parser = OptionParser()
> >  parser.add_option('-v', '--verbose',
> >    dest='verbose',
> >    default=True,
> >    action='store_true',
> >    help='Verbose logging. (default: %default)')
> >  parser.add_option('--servers',
> >    dest='servers',
> >    default='localhost:2181',
> >    help='Comma-separated list of host:port pairs. (default: %default)')
> >  global options
> >  global args
> >  (options, args) = parser.parse_args()
> >
> >  if options.verbose:
> >    logger.setLevel(logging.DEBUG)
> >  else:
> >    logger.setLevel(logging.INFO)
> >  formatter = logging.Formatter("%(asctime)s %(filename)s:%(lineno)d -
> > %(message)s")
> >  stream_handler = logging.StreamHandler()
> >  stream_handler.setFormatter(formatter)
> >  logger.addHandler(stream_handler)
> >
> >  zktest = ZKTest()
> >  zktest.daemon = True
> >  zktest.start()
> >
> >
> > if __name__ == '__main__':
> >  main()
> >
> >
> > Thanks!
> > Travis
> >
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679

Mime
View raw message