hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nkeywal (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads
Date Fri, 23 Mar 2012 10:37:28 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236499#comment-13236499

nkeywal commented on HBASE-5573:

bq. System.exit(1);
Actually is was already like that in hbasefsck, I replaced a tracker by a watcher that does
not watch to read the data, that's all.

bq. Why not add a create method to ZooKeeperWatcher that takes a name, conf, and Abortable?
Or is that a ZKW Constructor altogether?
Yes, the question is what to do when you're asked to abort. Here I reused the approach in
hbasefsck, just exit.

bq. N, can you explain more about what is going on here. How is it that we are not taking
a Watcher when we are creating a ZKW? Because we don't call start? (If so, that'd be 'elegant'

A ZKW is a watcher. When you create a ZKW, you create a RecoverableZooKeeper with yourself
as a parameter. Pseudo code is:
class RecoverableZooKeeper {
 ZooKeeper zk;
 RecoverableZooKeeper (Watcher w){ zk=new ZooKeeper(w) }

class ZooKeeperWatcher implements Watcher  {
 RecoverableZooKeeper rz;
 ZooKeeperWatcher (){ rz = new RecoverableZooKeeper(this); }

Using 'this' in a constructor is looking for problems but it works in this case (remember,
that's the existing code, not mine :-) ). Basically all these classes are very strongly coupled.
When I tried to partially decouple them it exploded in my hands because you anyway need a
watcher to manage the session expiry stuff. I don't have a middle solution here: it's either
a full rewriting with a lot of fun to keep the existing interfaces for backward compatibility
or nothing.

So in the final patch I've just done some cleanup (removed the last usage of getZooKeeperWatcher)
and the usage of any watcher. So there's no proof in the code, just that actually all the
functions we use on the client don't use a watcher. Anyway, they have a session in the ZK
servers so they are expensive. But thanks to #5399 the session on ZK will be closed after
5 minutes. So if you have an architecture with clients coming up and down, you will be able
to increase the number of clients. 

Three last comments:
- one of the design issue is that there ate two API: you can use directly any of the ZKW,
RZK, RK object or you can go through the static ZKUtils. May be the intermediate solutions
lie around this area.
- even if the existing design should not be shown to innocent scholars it's not that terrible,
because it's small. I didn't really like my first patches because I was adding more classes
and complexity without fixing the design. 
- On the long term, I think that it actually make sense to have a watcher in the client. It's
not about the previous code: The previous code was not really using watchers. The previous
code was setting watchers without using them. The new code (after #5399 and #5573) does not
use or set watchers. But when you have a fat client architecture like we have, it makes sense
to share some global state information, and it scales better when the info is pushed vs. pulled.
Having said that, there are many questions left: possible issues in how expensive it is with
ZooKeeper today, may be ZooKeeper is not really designed for this (it's not really a global
coordination work, as the client would be readers only) and so on. FWIW, it seems that the
current limit is around 10K sessions in ZK:

Patrick Hunt / Nov 18, 2010; 8:57pm
Re: number of clients/watchers

fyi: I haven't heard of anyone running over 10k sessions. I've tried 20k before and had issues.

A session is represented by a "ZooKeeper" object. One session per object. So if you have 10
client hosts each creating it's own ZooKeeper instance you'll have 10 sessions. This is regardless
of the number of znodes, watches, etc... Watches were designed to be lightweight and you can
maintain a large number of them. (25million spread across 500 sessions in my example)

There were also a discussion on ZK mailing list about lightweith sessions. http://markmail.org/message/cyow2xkneh2t3juc

> Replace client ZooKeeper watchers by simple ZooKeeper reads
> -----------------------------------------------------------
>                 Key: HBASE-5573
>                 URL: https://issues.apache.org/jira/browse/HBASE-5573
>             Project: HBase
>          Issue Type: Improvement
>          Components: client, zookeeper
>    Affects Versions: 0.96.0
>            Reporter: nkeywal
>            Assignee: nkeywal
>            Priority: Minor
>         Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 5573.v6.patch
> Some code in the package needs to read data in ZK. This could be done by a simple read,
but is actually implemented with a watcher. This holds ZK resources.
> Fixing this could also be an opportunity to remove the need for the client to provide
the master address and port.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message