zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ZOOKEEPER-2849) Quorum port binding needs exponential back-off retry
Date Thu, 16 Nov 2017 18:04:00 GMT

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16255706#comment-16255706
] 

ASF GitHub Bot commented on ZOOKEEPER-2849:
-------------------------------------------

GitHub user brian-lininger opened a pull request:

    https://github.com/apache/zookeeper/pull/419

    ZOOKEEPER-2849: Exponential back-off retry for Quorum port binding

      Added BackoffStrategy interface with ExponentialBackoffStrategy as a concrete implementation.
Enhanced
      QuorumCnxManager to utilize the provided BackoffStrategy to determine retry of port
binding as opposed to
      a fixed interval/count.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/brian-lininger/zookeeper ZOOKEEPER-2849

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zookeeper/pull/419.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #419
    
----
commit 0675e09a5d6122bff6b6543d80076b03d05d5f85
Author: Brian Lininger <brian.lininger@veeva.com>
Date:   2017-11-16T17:58:13Z

    ZOOKEEPER-2849:
      Added BackoffStrategy interface with ExponentialBackoffStrategy as a concrete implementation.
Enhanced
      QuorumCnxManager to utilize the provided BackoffStrategy to determine retry of port
binding as opposed to
      a fixed interval/count.

----


> Quorum port binding needs exponential back-off retry
> ----------------------------------------------------
>
>                 Key: ZOOKEEPER-2849
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2849
>             Project: ZooKeeper
>          Issue Type: Improvement
>          Components: quorum
>    Affects Versions: 3.4.6, 3.5.3
>            Reporter: Brian Lininger
>            Assignee: Brian Lininger
>            Priority: Minor
>
> Recently we upgraded the AWS instance type we use for running out ZooKeeper nodes, and
by doing so we're intermittently hitting an issue where ZooKeeper cannot bind to the server
election port because the IP is incorrect.  This is due to name resolution in Route53 not
being in sync when ZooKeeper starts on the more powerful EC2 instances.  Currently in QuorumCnxManager.Listener,
we only attempt to bind 3 times with a 1s sleep between retries, which is not long enough.
 
> I'm proposing to change this to follow an exponential back-off type strategy where each
failed attempt causes a longer sleep between retry attempts.  This would allow for Zookeeper
to gracefully recover when the host is misconfigured, and subsequently corrected, without
requiring the process to be restarted while also minimizing the impact to the running instance.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message