Return-Path: X-Original-To: apmail-zookeeper-user-archive@www.apache.org Delivered-To: apmail-zookeeper-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1A6BE930C for ; Wed, 21 Sep 2011 14:31:35 +0000 (UTC) Received: (qmail 11593 invoked by uid 500); 21 Sep 2011 14:31:34 -0000 Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org Received: (qmail 11521 invoked by uid 500); 21 Sep 2011 14:31:34 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 11378 invoked by uid 99); 21 Sep 2011 14:31:34 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Sep 2011 14:31:34 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ted.dunning@gmail.com designates 209.85.216.44 as permitted sender) Received: from [209.85.216.44] (HELO mail-qw0-f44.google.com) (209.85.216.44) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Sep 2011 14:31:28 +0000 Received: by qwb8 with SMTP id 8so4175478qwb.3 for ; Wed, 21 Sep 2011 07:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=3PlxzQIeUFVZZawtrSrjmlGf9wm3Gh0g6ZQGXpIIHaQ=; b=Qjo1+q8diVKPYIAobNPqyzS02S1evL2KaJY8JDbYwQHk/KI/RBfMEoM3K08R0O06mS WdBLXs+pBsbUu3UaKUFQ3Lvm+v6lu5FCtjli/mDLYEaZ8mN4uWvYfHXxHjZpnElWsibQ Ak0IHpyD4/JyzqqjmKwq65tUeQ7VX2bVsOiyc= Received: by 10.224.212.8 with SMTP id gq8mr682304qab.377.1316615467154; Wed, 21 Sep 2011 07:31:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.224.89.13 with HTTP; Wed, 21 Sep 2011 07:30:47 -0700 (PDT) In-Reply-To: <69D3016305F9084FBD2C4A0DF189BD5C1777E427BB@GSCMAMP02EX.firmwide.corp.gs.com> References: <69D3016305F9084FBD2C4A0DF189BD5C1777E427BB@GSCMAMP02EX.firmwide.corp.gs.com> From: Ted Dunning Date: Wed, 21 Sep 2011 07:30:47 -0700 Message-ID: Subject: Re: Creating a znode with SEQUENTIAL_EPHEMERAL mode becomes corrupt in case of unstable network To: user@zookeeper.apache.org Cc: "dev@zookeeper.apache.org" Content-Type: multipart/alternative; boundary=20cf300fb4a939be7704ad746d3f X-Virus-Checked: Checked by ClamAV on apache.org --20cf300fb4a939be7704ad746d3f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable If you cannot tolerate this sort of situation, then the only solution is typically to avoid sequential ephemerals. The problem is that in the presence of a flaky network you cannot always tell if a failed create actually created the znode in question. This is because the network may have failed after the create succeeded, but before you got the result. In that case, since this is a sequential ephemeral, you can't know if your fil= e got created because you don't even know the name. Moreover, scanning doesn't help because if you could scan, you probably could have used a fixe= d unique name in the first place. There is a very long standing proposed (nearly complete) solution for this that requires some difficult coding. See https://issues.apache.org/jira/browse/ZOOKEEPER-22 2011/9/21 Fournier, Camille F. > This is expected. In cases where the network becomes unstable, it is the > responsibility of the client writer to handle disconnected events > appropriately and check to verify whether nodes they tried to write aroun= d > the time of these events did or did not succeed. It makes writing a > "Generic" client for ZK very difficult (search the mailing list for zkcli= ent > and you'll read a bunch of convos around this topic). Fortunately, many > things that rely on EPHEMERAL_SEQUENTIAL nodes can tolerate some duplicat= ion > of data, so often it's not a huge problem. > > C > > -----Original Message----- > From: =EB=B0=95=EC=98=81=EA=B7=BC(Alex) [mailto:alex.park@nexr.com] > Sent: Wednesday, September 21, 2011 9:16 AM > To: dev@zookeeper.apache.org > Cc: user@zookeeper.apache.org > Subject: Creating a znode with SEQUENTIAL_EPHEMERAL mode becomes corrupt = in > case of unstable network > > Hi, All > > I met a problem in creating a znode with SEQUENTIAL_EPHEMERAL mode under > unstable network condition. > > While a client did not receive a message that a sequential node was > created, > the ensemble has the znode, which is checked at zookeeper dashboard( > https://github.com/phunt/zookeeper_dashboard). > > If the client receives a DISCONNECTED event, it tries to reconnect. > Session timeout is 30 seconds. > > Unstable network condition is made as the following: > > The grinder agent sends a request of creating a znode of > CreateMode. SEQUENTIAL_EPHEMERAL. > ZK ensemble has three servers. > Each NIC of server is down and up repeatedly; > NIC of server1 become down every one minute and sleeps for 9 seconds, the= n > up > NIC of server2 become down every 2 minute and sleeps for 9 seconds, then = up > NIC of server3 become down every 3 minute and sleeps for 9 seconds, then = up > > Is there any idea or related issue? > > Thanks in advance. > > Alex > --20cf300fb4a939be7704ad746d3f--