curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Lewis <MLe...@nephilaadvisors.co.uk>
Subject RE: How to avoid CuratorConnectionLossException on leader loss?
Date Wed, 23 Sep 2015 08:39:16 GMT
Hi,

I was just wondering if anyone on the list had used the Nirmata library with Curator, it looks
ideal for a use-case I have.
the only thing I’m trying to find is an example that shows task distribution to different
services. It is isn’t obvious to me
how to specify the number of different processes/machines you want the tasks distributed to,
the example looks fine
for a workflow running on one process.

Thanks in advance,
Mike Lewis


From: Jordan Zimmerman [mailto:jordan@jordanzimmerman.com]
Sent: 14 September 2015 14:49
To: Jens Rantil
Cc: user@curator.apache.org
Subject: Re: How to avoid CuratorConnectionLossException on leader loss?

Correct. You can see this in many of Curator’s unit tests.

-Jordan




On September 14, 2015 at 4:31:30 AM, Jens Rantil (jens.rantil@tink.se<mailto:jens.rantil@tink.se>)
wrote:
Hi Jordan,

Thanks for taking your time to answer me. A follow-up question: If things are properly configured,
I should expect Curator to transparently fail over to the new Zookeeper leader without my
application code noticing it, right?

Thanks,
Jens

On Sun, Sep 13, 2015 at 10:37 PM, Jordan Zimmerman <jordan@jordanzimmerman.com<mailto:jordan@jordanzimmerman.com>>
wrote:
Curator will only retry until the connection timeout and/or retry policy gives up. Try increasing
your connection timeout and allow more than 3 retries.

-Jordan




On September 13, 2015 at 11:16:07 AM, Jens Rantil (jens.rantil@tink.se<mailto:jens.rantil@tink.se>)
wrote:
Dear Curator(s),

A couple of days ago we did some maintenance of our Zookeeper ensemble and did a rolling restart
of each node. Restarting the followers worked like a charm. However, restarting leader started
throwing/logging CuratorConnectionLossException exceptions that trickled down to our application
code until a reelection had occured. Example:

https://gist.github.com/JensRantil/309fa1bf17ee2982b8e7

We were hoping that Curator would gracefully retry until a leader had been reelected, but
I'm sure there is something we need to tweak for this to avoid happening again.

Question: To avoid this to happen in the future, should we simply increase our retry policy
to retry longer before giving up?

Additional information:

  *   Zookeeper version 1.4.5
  *   Curator version 2.7.0
  *   We are currently using the following retrying policy: new ExponentialBackoffRetry(1000,
3);
  *   Zookeeper configuration all default except initLimit=60 and syncLimit=30.
Thanks,
Jens

--
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se<mailto:jens.rantil@tink.se>
Phone: +46 708 84 18 32
Web: www.tink.se<http://www.tink.se/>

Facebook<https://www.facebook.com/#!/tink.se> Linkedin<http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
Twitter<https://twitter.com/tink>



--
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se<mailto:jens.rantil@tink.se>
Phone: +46 708 84 18 32
Web: www.tink.se<http://www.tink.se/>

Facebook<https://www.facebook.com/#!/tink.se> Linkedin<http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
Twitter<https://twitter.com/tink>

--------------------------------------------------------------------------------------------------------------------------
This email has been sent to you on behalf of Nephila Advisors LLC (“Advisors”). Advisors
provides consultancy services to Nephila Capital Ltd. (“Capital”), an investment advisor
managed and carrying on business in Bermuda. Advisors and its employees do not act as agents
for Capital or the funds it advises and do not have the authority to bind Capital or such
funds to any transaction or agreement.

The information in this e-mail, and any attachment therein, is confidential and for use by
the addressee only. Any use, disclosure, reproduction, modification or distribution of the
contents of this e-mail, or any part thereof, other than by the intended recipient, is strictly
prohibited. If you are not the intended recipient, please return the e-mail to the sender
and delete it from your computer. This email is for information purposes only, nothing contained
herein constitutes an offer to sell or buy securities, as such an offer may only be made from
a properly authorized offering document. Although Nephila attempts to sweep e-mail and attachments
for viruses, it does not guarantee that either are virus-free and accepts no liability for
any damage sustained as a result of viruses.
--------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------
This email has been sent to you on behalf of Nephila Advisors UK (“Advisors UK”). Advisors
UK provides consultancy services to Nephila Capital Ltd. (“Capital”), an investment advisor
managed and carrying on business in Bermuda. Advisors UK and its employees do not act as agents
for Capital or the funds it advises and do not have the authority to bind Capital or such
funds to any transaction or agreement.

The information in this e-mail, and any attachment therein, is confidential and for use by
the addressee only. Any use, disclosure, reproduction, modification or distribution of the
contents of this e-mail, or any part thereof, other than by the intended recipient, is strictly
prohibited. If you are not the intended recipient, please return the e-mail to the sender
and delete it from your computer. This email is for information purposes only, nothing contained
herein constitutes an offer to sell or buy securities, as such an offer may only be made from
a properly authorized offering document. Although Nephila attempts to sweep e-mail and attachments
for viruses, it does not guarantee that either are virus-free and accepts no liability for
any damage sustained as a result of viruses.
--------------------------------------------------------------------------------------------------------------------------
Mime
View raw message