Return-Path: X-Original-To: apmail-curator-user-archive@minotaur.apache.org Delivered-To: apmail-curator-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DE52E18EC0 for ; Wed, 14 Oct 2015 23:05:26 +0000 (UTC) Received: (qmail 43320 invoked by uid 500); 14 Oct 2015 23:05:26 -0000 Delivered-To: apmail-curator-user-archive@curator.apache.org Received: (qmail 43271 invoked by uid 500); 14 Oct 2015 23:05:26 -0000 Mailing-List: contact user-help@curator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@curator.apache.org Delivered-To: mailing list user@curator.apache.org Received: (qmail 43258 invoked by uid 99); 14 Oct 2015 23:05:26 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Oct 2015 23:05:26 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 4827DC446A for ; Wed, 14 Oct 2015 23:05:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.901 X-Spam-Level: ** X-Spam-Status: No, score=2.901 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id sD3Usa-zkryX for ; Wed, 14 Oct 2015 23:05:08 +0000 (UTC) Received: from mail-yk0-f181.google.com (mail-yk0-f181.google.com [209.85.160.181]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id BF5D12326E for ; Wed, 14 Oct 2015 23:05:07 +0000 (UTC) Received: by ykoo7 with SMTP id o7so62240816yko.0 for ; Wed, 14 Oct 2015 16:05:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=YqtHK91ycN59y7FhK2rrSjzpa6fvLbvNfQxNyyezDmc=; b=Q6LvRBiGdjWL/wvofPSqkxPU3IEZ3k60Am6enpr0m579ZelGaiQyLCmhRBt0PMlBKA s2HAmvU2lpNSTgl/v5d0ziT+lzzmnYt1IQo1290gMZYwE5F8yBGQhLCLkXt0g/HSX5NZ wAoaJJTaCYhIRc/pjMfW/ug+mW7jxMYiQoXVlaPdhLwDsZm3IKau57PPpc5pKqF1lo87 tx699XKBL7OPpjBQAgz40WG7X0LdpkurPIxW6ne6zejLe8bLHkMnJ3UzbJpUOjsLGQ6F VEwV6WpUS9bhOgqpFsy7V6UnuJc5Y2IVWA5+a3aczTBsnIgSAhpb0oDDCKLsDRe6k44c qPIw== MIME-Version: 1.0 X-Received: by 10.129.153.134 with SMTP id q128mr4213668ywg.323.1444863907041; Wed, 14 Oct 2015 16:05:07 -0700 (PDT) Received: by 10.37.96.195 with HTTP; Wed, 14 Oct 2015 16:05:06 -0700 (PDT) In-Reply-To: References: <80EE66F5-3671-4605-A119-5CA8767498FC@jordanzimmerman.com> Date: Thu, 15 Oct 2015 10:05:06 +1100 Message-ID: Subject: Re: Persistent Ephemeral Node From: Cameron McKenzie To: user@curator.apache.org Content-Type: multipart/alternative; boundary=94eb2c0b7e9aed9ea10522189714 --94eb2c0b7e9aed9ea10522189714 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I guess you could work it out based on the ExponentialBackoffRetry algorithm, but it seems pretty ugly. Is it possible for you to instead just start up a new instance of the application immediately, and then if your crashed / seperated application comes back online, it can check if the ephemeral node is already present and if it is just shutdown? That would seem much less error prone. On Thu, Oct 15, 2015 at 9:59 AM, Vikrant Singh wrote: > Oh yes, I can do that. I can fix the path. > I create these node using default session time outs in curator framework.= . > I use ExponentialBackoffRetry with retry time out of 1s and retry count > of 29. > > So when I get a remove event for a PEM node... how long I should wait to > make sure that its client has given up on tries to rewrite it? > > On Wed, Oct 14, 2015 at 3:51 PM, Jordan Zimmerman < > jordan@jordanzimmerman.com> wrote: > >> Store your PEMs in a known path. Or, store a known value in the node. >> >> -JZ >> >> >> On Oct 14, 2015, at 5:49 PM, Vikrant Singh >> wrote: >> >> What I want is to differentiate between "ephemeral nodes" and >> "persistent ephemeral node". >> >> On Wed, Oct 14, 2015 at 3:20 PM, Cameron McKenzie > > wrote: >> >>> You could keep track of which nodes are ephemeral by looking at the sta= t >>> object passed to the NODE_ADDED event in the TreeCache. Then when you g= et a >>> NODE_REMOVED event you could check if it's one of your ephemeral nodes? >>> >>> On Thu, Oct 15, 2015 at 4:06 AM, Vikrant Singh < >>> vikrant.subscribe@gmail.com> wrote: >>> >>>> yes you got it correct... a slight correction. I am trying to find a >>>> way where I can make my tree cache event handler aware that "node remo= ved" >>>> event is coming from a persistent ephemeral node and delay any action= till >>>> the point the node's curator framework gives up on rewriting the node = to ZK. >>>> >>>> On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie < >>>> mckenzie.cam@gmail.com> wrote: >>>> >>>>> So, if I'm understanding correctly, if your application loses its >>>>> connection to ZooKeeper (or crashes), then a new instance will be sta= rted >>>>> in its place, and if the original instance reconnects then you don't = want >>>>> it to try and recreate its ephemeral node? >>>>> >>>>> If that's the case, then I think that you need additional logic beyon= d >>>>> what a PersistentEphemeralNode is going to provide you, as you will n= eed to >>>>> check if the node already exists on reconnection. >>>>> >>>>> On Tue, Oct 13, 2015 at 8:43 AM, Vikrant Singh < >>>>> vikrant.subscribe@gmail.com> wrote: >>>>> >>>>>> yeah I considered that.. but I ended up using tree cache because it >>>>>> gives me more control on the state of tree . I have some custom hand= lers >>>>>> for add/remove/update events. Same cache is also used for service di= scovery. >>>>>> >>>>>> On Mon, Oct 12, 2015 at 2:33 PM, David Kesler >>>>>> wrote: >>>>>> >>>>>>> There=E2=80=99s also already a Curator service discovery extension = library >>>>>>> that you can look at: >>>>>>> http://curator.apache.org/curator-x-discovery/index.html. It=E2=80= =99s >>>>>>> basically boiling down to the same strategy of sticking an ephemera= l node >>>>>>> into ZK, but with some additional convenience and functionality bui= lt on >>>>>>> top of it. >>>>>>> >>>>>>> >>>>>>> >>>>>>> *From:* Cameron McKenzie [mailto:mckenzie.cam@gmail.com] >>>>>>> *Sent:* Monday, October 12, 2015 5:05 PM >>>>>>> *To:* user@curator.apache.org >>>>>>> *Subject:* Re: Persistent Ephemeral Node >>>>>>> >>>>>>> >>>>>>> >>>>>>> hey Vikrant, >>>>>>> >>>>>>> Using a persistent ephemeral node just means that your application >>>>>>> code doesn't need to worry about handling recreation of the node wh= en it >>>>>>> reconnects to ZooKeeper after connection / session loss. >>>>>>> >>>>>>> >>>>>>> >>>>>>> If your ephemeral node should always be present whenever your >>>>>>> application instance is running, then this would be a good use case= for a >>>>>>> persistent ephemeral node. >>>>>>> >>>>>>> cheers >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh < >>>>>>> vikrant.subscribe@gmail.com> wrote: >>>>>>> >>>>>>> I have some basic question on persistent ephemeral node. >>>>>>> >>>>>>> Here is some background... >>>>>>> >>>>>>> >>>>>>> >>>>>>> We have a zoo keeper based service discovery setup. Each service >>>>>>> register itself as a ephemeral node with zookeeper.When a service g= o down >>>>>>> its ephemeral node is removed from zookeeper and we know that serv= ice is >>>>>>> down and we provision it again. >>>>>>> >>>>>>> At present we create plain ephemeral node. I am wondering what >>>>>>> benefit/risks we may get if move to persistent ephemeral ones. I s= ee one >>>>>>> problem... using plane ephemeral node we can rely on state of ZK t= o make a >>>>>>> decision like service is down. This is because we are sure that if = a node >>>>>>> get deleted with zoo keeper it will never comeback from same proces= s. But >>>>>>> if moved to "persistent ephemeral" I guess same may not be the case= . >>>>>>> >>>>>>> >>>>>>> >>>>>>> Please let me know what you think of the same. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Also I would like to know what are the best scenario where one >>>>>>> should prefer using persistent ephemeral node over ephemeral node. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Vikrant >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >> > --94eb2c0b7e9aed9ea10522189714 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I guess you could work it out based on the ExponentialBack= offRetry algorithm, but it seems pretty ugly. Is it possible for you to ins= tead just start up a new instance of the application immediately, and then = if your crashed / seperated application comes back online, it can check if = the ephemeral node is already present and if it is just shutdown?

<= /div>
That would seem much less error prone.

On Thu, Oct 15, 2015 at 9:59 AM,= Vikrant Singh <vikrant.subscribe@gmail.com> wrote= :
Oh yes, I can do that. I can fix th= e path.=C2=A0
I create these node using default session time outs in curator= framework.. I use=C2=A0ExponentialBackoffRetry with retry time out of 1s an= d retry count of 29.

So when I get a remove event for a= PEM node... how long =C2=A0I should wait to make sure that its client has = given up on tries to rewrite it?=C2=A0

On Wed, Oct 14, 2015 at 3:51 PM, Jordan Zimmerman <jorda= n@jordanzimmerman.com> wrote:
Store your PEMs in a known path. Or,= store a known value in the node.

-JZ
<= div>

On Oct 14, 2015, at 5:= 49 PM, Vikrant Singh <vikrant.subscribe@gmail.com> wrote:

What I =C2=A0want is to differentiate between "ephemeral= nodes" and "persistent ephemeral node".=C2=A0

On Wed, Oct 14, 2015 at 3= :20 PM, Cameron McKenzie <mckenzie.cam@gmail.com> wrote= :
You could keep track o= f which nodes are ephemeral by looking at the stat object passed to the NOD= E_ADDED event in the TreeCache. Then when you get a NODE_REMOVED event you = could check if it's one of your ephemeral nodes?

On Thu, Oct 15, 2015 at = 4:06 AM, Vikrant Singh <vikrant.subscribe@gmail.com> wrote:
yes you got it= correct... a slight correction. I am trying to find a way where I can make= my tree cache event handler aware that "node removed" event is c= oming from a persistent ephemeral node =C2=A0and delay any action till the = point the node's curator framework gives up on rewriting the node to ZK= .

= On Mon, Oct 12, 2015 at 3:38 PM, Cameron McKenzie <mckenzie.cam@gmail= .com> wrote:
So, if I'm understanding correctly, if your application loses its co= nnection to ZooKeeper (or crashes), then a new instance will be started in = its place, and if the original instance reconnects then you don't want = it to try and recreate its ephemeral node?

If that's= the case, then I think that you need additional logic beyond what a Persis= tentEphemeralNode is going to provide you, as you will need to check if the= node already exists on reconnection.

On Tue, Oct 13, 2015 at 8:43 AM, = Vikrant Singh <vikrant.subscribe@gmail.com> wrote:=
yeah I considered that.= . but I ended up using tree cache because it gives me more control on the s= tate of tree . I have some custom handlers for add/remove/update events. Sa= me cache is also used for service discovery.

On Mon, Oct 12, 2015 at 2:33 PM,= David Kesler <DKesler@yodle.com> wrote:

There=E2=80=99s also= already a Curator service discovery extension library that you can look at= :=C2=A0 http://curator.apache.org/curator-x-discovery/index.html.= =C2=A0 It=E2=80=99s basically boiling down to the same strategy of sticking= an ephemeral node into ZK, but with some additional convenience and functionality built on top of it.=C2=A0

=C2=A0

From: Cameron Mc= Kenzie [mailto:= mckenzie.cam@gmail.com]
Sent: Monday, October 12, 2015 5:05 PM
To: use= r@curator.apache.org
Subject: Re: Persistent Ephemeral Node

=

=C2=A0

hey Vikrant,

Using a persistent ephemeral node just means th= at your application code doesn't need to worry about handling recreatio= n of the node when it reconnects to ZooKeeper after connection / session lo= ss.

=C2=A0

If your ephemeral node should always be present= whenever your application instance is running, then this would be a good u= se case for a persistent ephemeral node.

cheers

=C2=A0

=C2=A0

On Tue, Oct 13, 2015 at 6:03 AM, Vikrant Singh = <vikran= t.subscribe@gmail.com> wrote:

I have some basi= c question on persistent ephemeral node.=C2=A0

Here is some background...

=C2=A0

We have a zoo keeper based service discovery se= tup. Each service register itself as a ephemeral node with zookeeper.When a= service go down =C2=A0its ephemeral node is removed from zookeeper and we = know that service is down and we provision it again.=C2=A0

At present we create plain ephemeral node. I am wondering what benefit/risk= s we may get if move to persistent ephemeral ones.=C2=A0 I see one problem.= .. using=C2=A0 plane ephemeral node we can rely on state of ZK to make a de= cision like service is down. This is because we are sure that if a node get deleted with zoo keeper it will never comeb= ack from same process. But if moved to "persistent ephemeral" I g= uess same may not be the case.

=C2=A0

Please let me know what you think of the same.<= u>

=C2=A0

Also I would like to know what are the best sce= nario where one should prefer using persistent ephemeral node over ephemera= l node.=C2=A0

=C2=A0

Thanks,

Vikrant

=C2=A0









--94eb2c0b7e9aed9ea10522189714--