Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
MIME-Version: 1.0
In-Reply-To: <BLUPR15MB003325C9257D8A359A241C15B9740@BLUPR15MB0033.namprd15.prod.outlook.com>
References: <CAFbh0Q1Yc3fwQpufC6sO8Q0no2sBQMv57u5p6hidrc=Tg-c_8w@mail.gmail.com>
 <BLUPR15MB003325C9257D8A359A241C15B9740@BLUPR15MB0033.namprd15.prod.outlook.com>
From: Michael Han <hanm@cloudera.com>
Date: Wed, 25 Jan 2017 13:33:18 -0800
Message-ID: <CA+i0x1+vJXejHFCF-y90WQ9-8QxoV8LPca2t1XbZ=WsbVpFw8A@mail.gmail.com>
Subject: Re: are ephemeral nodes removed when client receives session expiration
To: UserZooKeeper <user@zookeeper.apache.org>
Content-Type: multipart/alternative; boundary=001a114e665efbbf860546f1fca0
archived-at: Wed, 25 Jan 2017 21:33:56 -0000

--001a114e665efbbf860546f1fca0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

>> Does ZK guarantee that ephemeral nodes from a client are removed on the
sever by the time the client receives a session expiration event?

"the server" is a vague definition, as ZooKeeper ensemble is composed of
multiple servers :).

>> Therefore, it seems to be possible for a client to connect to another
server to see the node there.

This seems the only case I can think of that lead to the inconsistent view
from client side. I'll elaborate as follows, first the guarantees of
ZooKeeper that's relevant to this case:

* ZooKeeper quorum should have already committed the transaction of closing
the session when a client receives the session expire event.

* Clean up of ephemeral nodes associated with the session is part of the
closing session transaction, so for the quorum of servers who have already
committed the transaction, the ephemeral nodes should have gone already, on
those servers.

* ZooKeeper quorum would not have processed the new session establishment
request for the same client, until after the closing session request has
been processed because transactions are ordered across quorum.

Given these guarantees, if a client reestablishes a new session via
connecting to a server which was the quorum of servers that committed the
closing session transaction, then the client should not see the old
ephemeral node upon new session established.

ZooKeeper does not guarantee a write transaction occur synchronously across
all of the servers, since a write request only requires a quorum of servers
to acknowledge. As a result, it is valid that some servers might lag behind
the state of the quorum. I suspect this case is possible:

* Client receives session expire event, and client close its connection to
server A.

* Client reconnects to server B, which lags behind quorum, that does not
contain the changes to the data tree regarding ephemeral nodes.

* Client sees the ephemeral node so it does nothing. Later the node is
cleaned up when server B sync with quorum.

Client can ensure it always see the state of truth of the quorum by issuing
a sync() request before issuing a read request. A sync request will force
the server it's connecting to sync with the quorum. If Kafka does this,
will the bug go away? Of course, retry creating ephemeral nodes can also
solve the problem (there are possible other solutions as well, by having
client to do some book keeping work to differentiate versions between
ephemeral nodes).


On Wed, Jan 25, 2017 at 11:32 AM, Ryan Zhang <yangzhangrice@hotmail.com>
wrote:

> Good question, AFAIK, it=E2=80=99s not the case.
>
> The server will throw an SessionExpiredException during checkSession call
> as soon as the session is marked as isClosing. However, session expiratio=
n
> actually requires a transaction (of type OpCode.closeSession) which will =
be
> send to the leader to go through the quorum.  The session and ephemeral
> node will only be removed after the transaction is  committed and process=
ed
> in the final processor on other nodes. Therefore, it seems to be possible
> for a client to connect to another server to see the node there. I am not
> entirely sure if it can use the same session id though, it seems possible
> as the session close is only based on the session expire time and there c=
an
> be delays in session pings.
>
> On Jan 25, 2017, at 8:53 AM, Jun Rao <junrao@gmail.com<mailto:junra
> o@gmail.com>> wrote:
>
> Hi,
>
> Does ZK guarantee that ephemeral nodes from a client are removed on the
> sever by the time the client receives a session expiration event? I am
> getting conflicting info on this (
> https://issues.apache.org/jira/browse/KAFKA-4277). Could someone clarify?
>
> Thanks,
>
> Jun
>
>


--=20
Cheers
Michael.

--001a114e665efbbf860546f1fca0--