Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Received-SPF: pass (nike.apache.org: domain of Camille.Fournier@gs.com
 designates 204.4.187.100 as permitted sender)
From: "Fournier, Camille F." <Camille.Fournier@gs.com>
To: "'user@zookeeper.apache.org'" <user@zookeeper.apache.org>
Date: Thu, 4 Aug 2011 13:29:17 -0400
Subject: devops/admin/client question: What do you do when you rollback?
Thread-Topic: devops/admin/client question: What do you do when you rollback?
Thread-Index: AcxSzAi6NbDQzEfaRLGCBTDqT62UrQ==
Message-ID: 
 <69D3016305F9084FBD2C4A0DF189BD5C1770F901E0@GSCMAMP02EX.firmwide.corp.gs.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_69D3016305F9084FBD2C4A0DF189BD5C1770F901E0GSCMAMP02EXfi_"
MIME-Version: 1.0

--_000_69D3016305F9084FBD2C4A0DF189BD5C1770F901E0GSCMAMP02EXfi_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

We had an issue here the other day where the ZK servers were running poorly=
, and in an effort to get them healthy again we ended up rolling back the c=
luster state. While this was, in retrospect, not the right solution to the =
problem we were facing, it brought up another problem. Namely, that many of=
 our clients couldn't reconnect with their sessions because their zxid was =
too high (expected), but that the error they got when trying to do that rec=
onnection was just a vanilla disconnected error. The result was that most o=
f our clients had to be bounced.

Aside from trying hard to avoid ever rolling back the cluster state, does a=
nyone have a way they deal with this situation if it occurs? Should we cons=
ider enhancing the error message to the client so we could track the fact t=
hat we were ahead of the quorum zxid and react sensibly? Alternately, since=
 we were sending a sessionId along with the zxid, perhaps it would be nice =
to check to see if the sessionId exists before checking the zxid, which wou=
ld send an expired state signal which my client code could handle cleanly.

Any ideas or suggestions would be welcome.

C


--_000_69D3016305F9084FBD2C4A0DF189BD5C1770F901E0GSCMAMP02EXfi_--