curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imesha Sudasingha <imesha...@cse.mrt.ac.lk>
Subject Re: PathChildrenCache : Accuracy
Date Tue, 25 Oct 2016 04:03:18 GMT
I'm using zookeeper for a distributed command framework where all the
commands are written in znodes as persistent sequential nodes under the
same parent znode. In some occasions I have to make sure that the order of
the commands executed is same as the order they were published.

I went through the PathChildrenCache source code and found out that, it
guarantees that we get the events, but can be out of order if the
connection was disconnected and reconnected.

Thanks both of you for your valuable time. I now completely understand the
PathChildrenCache implementation. So, I'm also keeping track of the
CVersion in order to address the out of order issue.

Thanks!

Regards,
Imesha Sudasingha

On Oct 25, 2016 3:46 AM, "Cameron McKenzie" <mckenzie.cam@gmail.com> wrote:

> You're absolutely correct, I just completely confused myself. Imesha,
> ignore my previous comment!
> cheers
>
> On Tue, Oct 25, 2016 at 3:43 AM, Jordan Zimmerman <
> jordan@jordanzimmerman.com> wrote:
>
>> PathChildrenCache was never intended to retain the order of events. In my
>> opinion, relying on order of events in a ZooKeeper-based application is a
>> design mistake. PathChildrenCache is designed to provide an eventually
>> consistent view of a ZNode’s children.
>>
>> Imesha, can you explain why you need to rely on event ordering?
>>
>> -Jordan
>>
>> On Oct 24, 2016, at 11:20 AM, Scott Blum <dragonsinth@gmail.com> wrote:
>>
>> Are you sure?  I haven't dug deeply into PathChildrenCache but TreeCache
>> definitely does not work this way.  The act of re-registering the watcher
>> is what fetches data.  Let me amend your event list:
>>
>> -Watch children of /X
>> -Node /X/1 added
>> -TreeCache receives the ZK event for /X children changed
>> -Node /X/2 added (this event would be missed).
>> -TreeCache calls getChildren(/X) to re-register the watcher, sees both
>> /X/1 and /X/2
>> -TreeCache publishes events for both /X/1 and /X/2
>>
>> I looked at PathChildrenCache just a little and it looks like it might
>> work the same way.
>>
>>
>> IIRC, the kinds of events that might be missed are this:
>>
>> -Watch children of /X
>> -Node /X/1 added
>> -TreeCache receives the ZK event for /X children changed
>> -Node /X/2 added (this event would be missed).
>> -Node /X/2 deleted (this event would be missed).
>> -TreeCache calls getChildren(/X) to re-register the watcher, sees only
>> /X/1
>>
>> In this case, TreeCache would never know about the existence of /X/2
>> since it was created and deleted without the client being able to "see"
>> it.  This is a limitation in ZK's design.
>>
>> On Sun, Oct 23, 2016 at 10:44 PM, Cameron McKenzie <
>> mckenzie.cam@gmail.com> wrote:
>>
>>> Ordering should be consistent. There is always the possibility of
>>> missing a CHILD_CREATED event (or any other event), if it occurs in between
>>> the ZK client being notified of an event and before the client has
>>> reregistered the watch.
>>>
>>> i.e.
>>>
>>> -Watch children of /X
>>> -Node /X/1 added
>>> -Client is notified of /X/1 child created event
>>> -Node /X/2 added (this event would be missed).
>>> -Client reregisters watch for children on /X
>>>
>>>
>>> On Mon, Oct 24, 2016 at 1:15 PM, Imesha Sudasingha <
>>> imesha.13@cse.mrt.ac.lk> wrote:
>>>
>>>> Thank you for the response.
>>>>
>>>> Yes, I'm aware of the conditions of an eventually consistent system.
>>>> What I wanted to know was, is there any possibility for the
>>>> PathChildrenCache to emit CHILD_CREATED like events out of order and what
>>>> is the possibility of missing such event?
>>>>
>>>> Regards,
>>>> Imesha Sudasingha
>>>>
>>>> On Oct 21, 2016 7:12 PM, "Jordan Zimmerman" <jordan@jordanzimmerman.com>
>>>> wrote:
>>>>
>>>>> PathChildrenCache[1] is the solution you have provided to watch on a
>>>>> given node without using native zookeeper watchers.
>>>>>
>>>>>
>>>>> PathChildrenCache uses native watchers. Have a look at the source.
>>>>>
>>>>> Can anyone please clarify how the above effect can affect the accuracy
>>>>> of events listened?
>>>>>
>>>>>
>>>>> The point of that message is to remind you that ZooKeeper is an
>>>>> eventually consistent system. You are always seeing the view that the
>>>>> server you are connected to has. This is a feature of ZooKeeper, not
>>>>> Curator.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> -Jordan
>>>>>
>>>>> On Oct 21, 2016, at 4:44 AM, Imesha Sudasingha <
>>>>> imesha.13@cse.mrt.ac.lk> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I have been using apache zookeeper. Now I'm willing to switch to
>>>>> CuratorFramework as it contains many useful recipes inbuilt.
>>>>>
>>>>> PathChildrenCache[1] is the solution you have provided to watch on a
>>>>> given node without using native zookeeper watchers. As I went through
the
>>>>> API [1] documentation, the following ambiguous sentence has caused me
to
>>>>> think twice as I want consistency accuracy for my implementation (such
as
>>>>> not missing CHILD_CREATED events).
>>>>>
>>>>> "it's not possible to stay transactionally in sync. Users of this
>>>>> class must be prepared for false-positives and false-negatives.
>>>>> Additionally, always use the version number when updating data to avoid
>>>>> overwriting another process' change."
>>>>>
>>>>> Can anyone please clarify how the above effect can affect the accuracy
>>>>> of events listened? And is there a way to w atch on a given node without
>>>>> using Zookeeper watchers and PathChildrenCache?
>>>>>
>>>>> (PathChildrenCache has the functionality I required. But the above
>>>>> description in the API docs matters me)
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> [1] https://curator.apache.org/apidocs/org/apache/curator/fr
>>>>> amework/recipes/cache/PathChildrenCache.html
>>>>>
>>>>> Regards,
>>>>> Imesha Sudasingha
>>>>>
>>>>> --
>>>>> *Imesha Sudasingha*
>>>>> Undergraduate of Department of Computer Science and  Engineering,
>>>>> University of Moratuwa.
>>>>> +94717086160
>>>>> View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>
>>>>>
>>>>>
>>>>>
>>>
>>
>>
>

Mime
View raw message