hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7411) Use Netflix's Curator zookeeper library
Date Fri, 21 Dec 2012 05:35:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537691#comment-13537691
] 

stack commented on HBASE-7411:
------------------------------

Yeah our zk hookup is critical but it doesn't get much by way of attention given its import
(and yes, we need to do some dependency pruning)

It looks like dev on our 'curator', our custom zk client, is in a dead end and may be abandoned
going by whats is over in the fb java commons.  RecoverableZooKeeper became RecoveringZooKeeper
and it doesn't do stuff like write serialization id as first few bytes of the znode data as
ours does: https://github.com/facebook/jcommon/blob/master/zookeeper/src/main/java/com/facebook/zookeeper/RecoveringZooKeeper.java
 (We can ask the fb lads for the definitive).

Our zk interface is also currently a mess w/ code spread all about the place w/ a strange
layering that needs undoing (zookeeper package, ZKUtil, ZKAssign... clients import them all...
then there is callback handling distributed throughout).  We could with do a revisit hereabouts
in general.

Curator is used in more than just one project so has probably some extra resilience built
up.

It gets good reviews from zk fellas we all know and is under active development last time
I checked.

It does basic stuff like keeping the actual zookeeper instance from you so session can be
reestablished behind the scenes (we have actual zookeeper instance everywhere and are hosed
once session is gone... could entertain retries w/ curator as is, unless we did more code
on our part).

Implementing the reciepes is more new code in hbase w/ attendant bug fixing but if they are
already done in another open source project, after some vetting and test, why not use it instead
(as is, we need distributed locks, likely the double blocking receipe.... ).

Just saying.
                
> Use Netflix's Curator zookeeper library
> ---------------------------------------
>
>                 Key: HBASE-7411
>                 URL: https://issues.apache.org/jira/browse/HBASE-7411
>             Project: HBase
>          Issue Type: New Feature
>          Components: Zookeeper
>    Affects Versions: 0.96.0
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>
> We have mentioned using the Curator library (https://github.com/Netflix/curator) elsewhere
but we can continue the discussion in this.  
> The advantages for the curator lib over ours are the recipes. We have very similar retrying
mechanism, and we don't need much of the nice client-API layer. 
> We also have similar Listener interface, etc. 
> I think we can decide on one of the following options: 
> 1. Do not depend on curator. We have some of the recipes, and some custom recipes (ZKAssign,
Leader election, etc already working, locks in HBASE-5991, etc). We can also copy / fork some
code from there.
> 2. Replace all of our zk usage / connection management to curator. We may keep the current
set of API's as a thin wrapper. 
> 3. Use our own connection management / retry logic, and build a custom CuratorFramework
implementation for the curator recipes. This will keep the current zk logic/code intact, and
allow us to use curator-recipes as we see fit. 
> 4. Allow both curator and our zk layer to manage the connection. We will still have 1
connection, but 2 abstraction layers sharing it. This is the easiest to implement, but a freak
show? 
> I have a patch for 4, and now prototyping 2 or 3 whichever will be less painful. 
> Related issues: 
> HBASE-5547
> HBASE-7305
> HBASE-7212

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message