Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_BE0F85E4-5D11-4652-B23E-3715C65653D3"
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
Subject: Re: disconnected events and session expiration
From: Simon <cocoa@gmx.ch>
In-Reply-To: <etPan.55f4cf3c.5ea725f5.42a@Jordans-MacBook-Pro.local>
Date: Sun, 13 Sep 2015 13:28:52 +0200
Cc: user@zookeeper.apache.org
Message-Id: <DC13A95D-D9E3-4B31-B177-583A0679A543@gmx.ch>
References: <59E1893C-C885-4FD3-AED1-ED5976F74E08@gmx.ch>
 <etPan.55f4cf3c.5ea725f5.42a@Jordans-MacBook-Pro.local>
To: Jordan Zimmerman <jordan@jordanzimmerman.com>

--Apple-Mail=_BE0F85E4-5D11-4652-B23E-3715C65653D3
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

That didn=E2=80=99t really answer any of my questions.=20

If I own a lock, I am entitled to do some work exclusively. No one else =
should be doing that work. If I get disconnected or the session times =
out I have to stop working. Somebody else will take over the work in a =
short time. If I understood the programmers guide correctly, the expired =
event will not be delivered to me until I reconnect. Correct? So, I have =
to use the disconnected event to initiate a graceful stop. Stopping work =
might take some time, e.g. because I am doing a REST service call that =
takes up to 20s. Let=E2=80=99s say doing the call twice leads to data =
corruption in the backend service (e.g. HTTP POST, which is non =
idempotent). So, ideally, if I am still running, I should try my best to =
complete normally. If the state of the work units is kept in ZK, I =
cannot update the state anyway. If I store it in some other datastore, I =
might be able to update the state or not (depending on how the network =
has been partitioned).

The more I think about it, the harder it seems to get this stuff working =
reliably. What if my node crashes? I cannot complete my work normally. =
So, whoever takes over my work will try to redo it anyways. Either the =
receiver is made idempotent (which is not always possible) or the new =
work owner needs to be aware of the aborted task and be extra cautious, =
e.g. by checking whether the work unit has completed or not. It seems to =
me that making the =E2=80=9Ccrash=E2=80=9D case the default (i.e. =
=E2=80=9Ccrash=E2=80=9D the worker thread whenever a disconnected event =
is received) is the best solution. Then I am forced to make the crash =
case robust. Guess that=E2=80=99s what some people call =E2=80=9Ccrash-onl=
y design=E2=80=9D.

Simon


> On 13 Sep 2015, at 03:19 , Jordan Zimmerman =
<jordan@jordanzimmerman.com> wrote:
>=20
> I used to advise that people treat Disconnected the same as session =
loss as it=E2=80=99s safer. But, you can also set a timer when =
Disconnected is received and when your session timeout elapses you can =
then consider session loss (note, use the negotiated value from the ZK =
handle). FYI - version 3.0.0 of Apache Curator will have an option to =
choose this alternate method.
>=20
> -Jordan
>=20
>=20
>=20
> On September 12, 2015 at 4:47:46 PM, Simon (cocoa@gmx.ch =
<mailto:cocoa@gmx.ch>) wrote:
>=20
>> Hi=20
>>=20
>> I am trying to get a better understanding of Zookeeper and how it =
should be used. Let=E2=80=99s talk about the lock recipe =
(http://zookeeper.apache.org/doc/r3.4.6/recipes.html#sc_recipes_Locks). =20=

>>=20
>> - X aquires the lock=20
>> - X does some long running work (longer than the session timeout)=20
>> - X gets partioned away from the quorum while it was doing some work=20=

>> - after some time (determined by the timeout passed to ZK) Y will =
aquire the lock=20
>>=20
>> In that situation both X and Y are holding the lock (unless X is =
acting properly). If I understand the documentation correctly =
(http://zookeeper.apache.org/doc/r3.4.6/zookeeperProgrammers.html#ch_zkSes=
sions), X would receive a disconnected event in that situation (but not =
an expired event unless it successfully reconnects). So, X should stop =
doing the work it is doing until it gets reconnected. How much time does =
X have to stop the work it is doing? i.e. how long does it take from =
disconnected event sent to X to expiration of the ephemeral node used =
for the lock? Having two clients inside a critical section protected by =
a lock would not be a good idea.=20
>>=20
>> Regards,=20
>> Simon


--Apple-Mail=_BE0F85E4-5D11-4652-B23E-3715C65653D3--