Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\))
Subject: Re: Doubts about libzookeeper
From: Flavio Junqueira <fpj@apache.org>
In-Reply-To: 
 <CANcXBFNG9EOd-pkGog3+6PMggpOXoDjcRHYTZnzCH8kH_R17cw@mail.gmail.com>
Date: Tue, 4 Aug 2015 22:15:01 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <63702B46-DFBC-4FA3-BD13-E7C8ED3FD4E7@apache.org>
References: <2015080414393275398766@baidu.com>
 <CABF4AKgcAo8KHd1yPHzOF1UxNXwk3-xBSxzjzPJK8JhFa5L_pA@mail.gmail.com>
 <CABWqe2arGqz6NBXp7Pi_1t3nbMiWwWuLN2pPRRkXAWT6joS_Cw@mail.gmail.com>
 <CANcXBFMbaprMoEjTggv_T-Nc-Y4X4xVvrQc-3sZ5mwQGz_G_Og@mail.gmail.com>
 <CABWqe2bCk9LE9X5wU=WAh-LAQvmq-SBPFNjocMa2Gpd64kQ5gQ@mail.gmail.com>
 <CANcXBFOS0geazKw_oA=4pewRVgr9r9KaFK_1uy2Vp-f9DMS=Aw@mail.gmail.com>
 <CAKF1A=tz=OzqmcfTkBMYEuzJxC1h7s-0w=peOz8eZaJK3C80Fg@mail.gmail.com>
 <CANcXBFPB=VUyozHDcd-T9sHdnB+gMJLhrsiFYKUkSf+EsrwHgg@mail.gmail.com>
 <CABWqe2ZmLRoguf1s0Zpgyz1=c=o9qgS7EDnhwBM-Ym5Ktodg7g@mail.gmail.com>
 <CANcXBFNG9EOd-pkGog3+6PMggpOXoDjcRHYTZnzCH8kH_R17cw@mail.gmail.com>
To: user@zookeeper.apache.org

If the client isn't sure that the delete has gone through, just do it =
again once reconnected (to server 2 in the scenario described). Whatever =
response you get for the delete should determine what you need to do.=20

-Flavio

> On 04 Aug 2015, at 22:11, Alexander Shraer <shralex@gmail.com> wrote:
>=20
> maybe 1 or 2 synctime, is enough given what you said about syncs - =
after 1
> synctime
> we know that either server1 disconnected (and will have to bootstrap =
its
> state from the leader
> if it ever reconnects) or the request got to the leader. But since =
synctime
> may not be measured
> exactly from our request submission it maybe that 2 synctime are =
needed.
> Would need to look
> deeper into pings and synctime to tell for sure.
>=20
> On Tue, Aug 4, 2015 at 2:05 PM, Camille Fournier <camille@apache.org> =
wrote:
>=20
>> That's true. I spent some time trying to think about when and how =
that
>> would be possible, and didn't get very far. We have guarantees about =
how
>> far out of sync a quorum member can be before it's booted, so I would =
think
>> that there's some way to timebound this potentially to prevent it, a =
la
>> your suggestion about 3X synctime.
>>=20
>> C
>>=20
>>=20
>> On Tue, Aug 4, 2015 at 4:58 PM, Alexander Shraer <shralex@gmail.com>
>> wrote:
>>=20
>>> Yes, I checked and you're right. It gets queued at the leader until =
all
>>> previously proposed requests at the leader
>>> are committed. But still if the request is only on its way between
>> server 1
>>> and the leader sync won't immediately help, right ?
>>>=20
>>>=20
>>> On Tue, Aug 4, 2015 at 11:39 AM, Camille Fournier =
<camille@apache.org>
>>> wrote:
>>>=20
>>>> I thought that sync forced a flush of the queued events on a quorum
>>> member
>>>> before completing/got it in the path of events from the leader, so =
that
>>> it
>>>> won't return until all of the pending leader events before it have =
been
>>>> seen by this quorum member. Is that not correct?
>>>>=20
>>>> On Tue, Aug 4, 2015 at 2:20 PM, Alexander Shraer =
<shralex@gmail.com>
>>>> wrote:
>>>>=20
>>>>> It seems that since the delete may be in-flight (between server 1 =
and
>>>>> leader, or still being proposed by the leader)
>>>>> when the client connects to server 2, doing a sync right a way may
>> not
>>>> help
>>>>> since the operation hasn't been committed yet. Perhaps the client
>>> should
>>>>> wait some multiple of synclimit time (3x ?) before invoking the =
sync
>> to
>>>>> allow the delete to commit or disappear for sure. This is all =
related
>>> to
>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-22, which is still
>>> open
>>>>> unfortunately...
>>>>>=20
>>>>> On Tue, Aug 4, 2015 at 10:15 AM, Camille Fournier <
>> camille@apache.org>
>>>>> wrote:
>>>>>=20
>>>>>> True, I'm not sure when the xid increments. If that is the case,
>> you
>>>> can
>>>>>> force a sync before the read of the path, to prevent reading =
stale
>>>> data.
>>>>> So
>>>>>> that would be the solve for that edge case although it's an
>> expensive
>>>>>> solve.
>>>>>>=20
>>>>>> C
>>>>>>=20
>>>>>> On Tue, Aug 4, 2015 at 12:52 PM, Alexander Shraer <
>> shralex@gmail.com
>>>>=20
>>>>>> wrote:
>>>>>>=20
>>>>>>> Hi Camille,
>>>>>>>=20
>>>>>>> if the client received a response for the delete then sure it
>>>> shouldn't
>>>>>> be
>>>>>>> able to connect
>>>>>>> to servers that didn't see it. But if it disconnected before
>> seeing
>>>> the
>>>>>>> response the example seems possible to me.
>>>>>>> I haven't checked the code to see when exactly the transaction
>>> number
>>>>> is
>>>>>>> incremented at
>>>>>>> the client, so I may be wrong, but suppose for example that
>>>> zkserver-1
>>>>>>> crashes before
>>>>>>> sending the delete request to the leader. Then, the request is
>> gone
>>>>>>> forever. If you don't let the client
>>>>>>> connect to another server that hasn't seen the delete, the =
client
>>>> will
>>>>>>> never be able to connect.
>>>>>>> So it seems quite possible that it connects, then the request is
>>>>> executed
>>>>>>> (if zkserver-1 hasn't crashed
>>>>>>> after all) and the znode disappears.
>>>>>>>=20
>>>>>>> Alex
>>>>>>>=20
>>>>>>>=20
>>>>>>> On Tue, Aug 4, 2015 at 8:33 AM, Camille Fournier <
>>> camille@apache.org
>>>>>=20
>>>>>>> wrote:
>>>>>>>=20
>>>>>>>> ZooKeeper provides a session-coherent single system image
>>>> guarantee.
>>>>>> Any
>>>>>>>> request from the same session will see the results of all of
>> its
>>>>>> writes,
>>>>>>>> regardless of which server it connects to. See:
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>=20
>>>>>>=20
>>>>>=20
>>>>=20
>>>=20
>> =
http://zookeeper.apache.org/doc/r3.4.6/zookeeperProgrammers.html#ch_zkGuar=
antees
>>>>>>>>=20
>>>>>>>> So, if your session deletes, and the delete is successfully
>>>> processed
>>>>>> by
>>>>>>>> the quorum, you will not see the path that you have deleted no
>>>> matter
>>>>>>> what
>>>>>>>> server your session connects to. I believe in practice that
>> this
>>>>> means
>>>>>>> that
>>>>>>>> the ZK servers that might be behind your session (say server 2
>> is
>>>>>> lagging
>>>>>>>> behind a few commits) will refuse to allow your session to
>>> connect
>>>> to
>>>>>> it,
>>>>>>>> so that you will not see stale data.
>>>>>>>>=20
>>>>>>>> This means that the example Lokesh gave:
>>>>>>>>=20
>>>>>>>> "1. Quorum leader has forwarded request to zkserver-2 for
>> "delete
>>>>>> /path".
>>>>>>>> 2. If your client connects to "zkserver-2" after step 1 is
>>> executed
>>>>>> (get
>>>>>>>> /path). Then your "/path" will not be available.
>>>>>>>> 3. If your client connects to "zkserver-2" before step1 is
>>> executed
>>>>>> (get
>>>>>>>> /path) then your "/path" would be available and after some time
>>>> your
>>>>>> path
>>>>>>>> would not be available (after zkserver-2 is synched with the
>>>> leader)"
>>>>>>>>=20
>>>>>>>> Cannot happen, so long as you are in the same session.
>>>>>>>>=20
>>>>>>>> C
>>>>>>>>=20
>>>>>>>> On Tue, Aug 4, 2015 at 6:49 AM, Lokesh Shrivastava <
>>>>>>>> lokesh.shrivastava@gmail.com> wrote:
>>>>>>>>=20
>>>>>>>>> I think it depends on whether your request reaches zkserver-1
>>> and
>>>>>>> whether
>>>>>>>>> it is able to send the request to quorum leader. Considering
>>> that
>>>>>>> "delete
>>>>>>>>> /path" request has reached the quorum leader then following
>> may
>>>>>> happen
>>>>>>>>>=20
>>>>>>>>> 1. Quorum leader has forwarded request to zkserver-2 for
>>> "delete
>>>>>>> /path".
>>>>>>>>> 2. If your client connects to "zkserver-2" after step 1 is
>>>> executed
>>>>>>> (get
>>>>>>>>> /path). Then your "/path" will not be available.
>>>>>>>>> 3. If your client connects to "zkserver-2" before step1 is
>>>> executed
>>>>>>> (get
>>>>>>>>> /path) then your "/path" would be available and after some
>> time
>>>>> your
>>>>>>> path
>>>>>>>>> would not be available (after zkserver-2 is synched with the
>>>>> leader)
>>>>>>>>>=20
>>>>>>>>> Others can correct me if this is not how it works.
>>>>>>>>>=20
>>>>>>>>> Thanks.
>>>>>>>>> Lokesh
>>>>>>>>>=20
>>>>>>>>> On 4 August 2015 at 12:09, liangdong01@baidu.com <
>>>>>>> liangdong01@baidu.com>
>>>>>>>>> wrote:
>>>>>>>>>=20
>>>>>>>>>> Hi,
>>>>>>>>>>     I'm thinking about a program desgin with libzookeeper,
>>>> here
>>>>> is
>>>>>>> my
>>>>>>>>>> doubts:
>>>>>>>>>>=20
>>>>>>>>>>    1) first, I connnect to zkserver-1, and there exists
>> the
>>>> path
>>>>>>>>> "/path".
>>>>>>>>>>    2) I sends "delete /path", the request reaches(may
>> not, i
>>>>> don't
>>>>>>>> know
>>>>>>>>>> about that) zkserver-1 and dont't know whether this
>> effected,
>>>> and
>>>>>>> then
>>>>>>>>> lost
>>>>>>>>>> connection before response returns.
>>>>>>>>>>    3) reconnect the same session to zkserver-2,  and I
>> sends
>>>>> "get
>>>>>>>>> /path".
>>>>>>>>>>=20
>>>>>>>>>>    which one will the "get /path" return possibly :
>>>>>>>>>>    1, "not exists"
>>>>>>>>>>    2, "exists" and "always exists"
>>>>>>>>>>    3, "exists" and "not exists" afterwards
>>>>>>>>>>=20
>>>>>>>>>>    my biggist problem is wether the 3) will occur or not,
>>>>> thanks!
>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>>=20
>>>>>>>>>> liangdong01@baidu.com
>>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>=20
>>>>>>>=20
>>>>>>=20
>>>>>=20
>>>>=20
>>>=20
>>=20