hadoop-zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: ZKClient
Date Wed, 05 May 2010 17:06:23 GMT
While I agree DS is hard, I don't think we should lose the useful 
feedback given by Jonathan/Adam - that getting started with ZK is 
challenging and can be frustrating. We need to learn from this feedback 
and create some action items to address. One of the main things I've 
heard so far that we can act on today is that we should add 
examples/docs to round things out. I agree with this. Also the recipes 
page should be updated to point to the recipe implementations we 
recently added to the release.

One suggestion, it's much easier for new contributors/users to 
contribute to the examples than it is to jump into ZK core development. 
New users feel the pain most directly (recently), I'd encourage you to 
contribute back by creating an example or two. I'm sure the existing 
contributors would be happy to work with you to get them committed and 
released.

Regards,

Patrick

On 05/04/2010 03:43 PM, Ted Dunning wrote:
> Creating recipes is a great thing, but that doesn't change the fact that
> distributed systems are inherently a bit tricky, especially if you start
> with the assumption (as many people do) that Peter Deutsch was wrong.
>
> One of the great contributions of MapReduce style parallelism or the java
> concurrent package is that it provides safe trails in a pretty scary forest.
>   Good Zookeeper recipes could provide similar guidance with similar positive
> effects.
>
> On Tue, May 4, 2010 at 3:24 PM, Adam Rosien<adam@rosien.net>  wrote:
>
>> I'll check it out, but it is repeated in this list and on the web site
>> that it's not as easy as it seems. I just want to enumerate the
>> failure points and create abstractions to avoid them.
>>
>> .. Adam
>>
>> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar<mahadev@yahoo-inc.com>
>> wrote:
>>> Hi Adam,
>>>   I don't think zk is very very hard to get right. There are exmaples in
>>> src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
>> to
>>> make it even more easier for application to use.
>>>
>>> Regarding re registration of watches, you can deifnitely write code and
>>> submit is as a part of well documented contrib module which lays out the
>>> assumptions/design of it. It could very well be useful for others. Its
>> just
>>> that folks havent had much time to focus on these areas as yet.
>>>
>>> Thanks
>>> mahadev
>>>
>>>
>>> On 5/4/10 2:58 PM, "Adam Rosien"<adam@rosien.net>  wrote:
>>>
>>>> I use zkclient in my work at kaChing and I have mixed feelings about
>>>> it. On one hand it makes "easy things easy" which is great, but on the
>>>> other hand I very few ideas what assumptions it makes "under the
>>>> hood". I also dislike some of the design choices such as unchecked
>>>> exceptions, but that's neither here nor there. It would take some
>>>> extensive documentation work by the authors to really enumerate the
>>>> model and assumptions, but the project doesn't seem to be active
>>>> (either from it being adequate for its current users or just
>>>> inactive). I'm not sure I could derive the assumptions myself.
>>>>
>>>> I'm a bit frustrated that zk is "very, very hard to really get right".
>>>> At a project level, can't we create structures to avoid most of these
>>>> errors? Can there be a "standard model" with detailed assumptions and
>>>> implementations of all the recipes? How can we start this? Is there
>>>> something that makes this too hard?
>>>>
>>>> I feel like a recipe page is a big fail; wouldn't an example app that
>>>> uses locks and barriers be that much more compelling?
>>>>
>>>> For the common FAQ items like "you need to re-register the watch",
>>>> can't we just create code that implements this pattern? My goal is to
>>>> live up to the motto: a good API is impossible to use incorrectly.
>>>>
>>>> .. Adam
>>>>
>>>> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning<ted.dunning@gmail.com>
>> wrote:
>>>>> In general, writing this sort of layer on top of ZK is very, very hard
>> to
>>>>> get really right for general use.  In a simple use-case, you can
>> probably
>>>>> nail it but distributed systems are a Zoo, to coin a phrase.  The
>> problem is
>>>>> that you are fundamentally changing the metaphors in use so assumptions
>> can
>>>>> come unglued or be introduced pretty easily.
>>>>>
>>>>> One example of this is the fact that ZK watches *don't* fire for every
>>>>> change but when you write listener oriented code, you kind of expect
>> that
>>>>> they will.  That makes it really, really easy to introduce that
>> assumption
>>>>> in the heads of the programmer using the event listener library on top
>> of
>>>>> ZK.  Another example is how the atomic get content/set watch call works
>> in
>>>>> ZK is easy to violate in an event driven architecture because the
>> thread
>>>>> that watches ZK probably resets the watch.  If you assume that the
>> listener
>>>>> will read the data, then you have introduced a timing mismatch between
>> the
>>>>> read of the data and the resetting of the watch.  That might be OK or
>> it
>>>>> might not be.  The point is that these changes are subtle and tricky
to
>> get
>>>>> exactly right.
>>>>>
>>>>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway<
>>>>> jonathan.holloway@gmail.com>  wrote:
>>>>>
>>>>>> Is there any reason why this isn't part of the Zookeeper trunk
>> already?
>>>>>>
>>>>>
>>>
>>>
>>
>

Mime
View raw message