zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From TyqITstudent <...@git.apache.org>
Subject [GitHub] zookeeper pull request #687: Zookeeper 3169
Date Sun, 04 Nov 2018 09:21:58 GMT
GitHub user TyqITstudent opened a pull request:

    https://github.com/apache/zookeeper/pull/687

    Zookeeper 3169

    **Problem:**
    1. Sometimes Zookeeper cluster will receive a lot of connections from clients, sometimes
connection number even exceeds 1W. When zxid rolls over, the clients will reconnect and revalidate
the session.
    2. In Zookeeper design structure, when follower server receives the session revalidation
requests, it will send requests to leader server, which is designed to be responsible for
session revalidation.
    3. In a short time, Leader will handle lots of requests. I use a tool to get the statistics,
some clients need to wait over 20s. It is too long for some special clients, like ResourceManager.
    
    **Solution:**
    1. When zookeeper cluster finishes reelection(which will cost a few seconds). The leader
will send the time point **TimeA** to followers. (which is the approximate value of roll
over)
    
    2.  Followers can judge the most session revalidations.  When the timeout of the session
is less than **currentTime - timeA** , follower will put the session on the touchTable. (Every
half tickTime, followers will send sessions of touchTable to leader to validate).
    
    3.  When the timeout of the session is larger than **currentTime - timeA**, the follower
will send session revalidation request to leader right away.
    
    So the leader will receive fewer requests from followers.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/TyqITstudent/zookeeper ZOOKEEPER-3169

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zookeeper/pull/687.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #687
    
----
commit 620f9f4c22192e17d0fb9a4bb5486dcd2d8db2bd
Author: Flavio Paiva Junqueira <fpj@...>
Date:   2014-11-29T15:55:50Z

    ZOOKEEPER-2064 Prevent resource leak in various classes (Ted Yu via fpj)
    
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1642441 13f79535-47bb-0310-9956-ffa450edef68

commit a9718779da86d9e61e2432e6e87c031db311f42f
Author: Michi Mutsuzaki <michim@...>
Date:   2015-02-11T07:01:42Z

    ZOOKEEPER-1949 recipes jar not included in the distribution package (Rakesh R via michim)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1658890 13f79535-47bb-0310-9956-ffa450edef68

commit 7b3ba5fca5802cec37605f8099c6fdead6d3c872
Author: Michi Mutsuzaki <michim@...>
Date:   2015-02-22T21:11:11Z

    ZOOKEEPER-2114 jute generated allocate_* functions are not externally visible (Tim Crowder
via michim)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1661562 13f79535-47bb-0310-9956-ffa450edef68

commit 415bcd091228caef78e0d8b7bbdbbe67c48dee2f
Author: Michi Mutsuzaki <michim@...>
Date:   2015-02-22T22:08:16Z

    ZOOKEEPER-2073 Memory leak on zookeeper_close (Dave Gosselin via michim)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1661573 13f79535-47bb-0310-9956-ffa450edef68

commit 2c271faa1517ddaca0747cdf8760a1fb670f955f
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-04-07T00:50:58Z

    ZOOKEEPER-2056 Zookeeper 3.4.x and 3.5.0-alpha is not OSGi compliant (Deiwin Sarjas via
rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1671716 13f79535-47bb-0310-9956-ffa450edef68

commit 87fc5e19bfae81508de96b4bb86d0aa1d5374c1c
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-04-18T00:47:56Z

    ZOOKEEPER-2146 BinaryInputArchive readString should check length before allocating memory
(Hongchao Deng via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1674424 13f79535-47bb-0310-9956-ffa450edef68

commit 7c3fe9d224e318a842b27113b7594cc7a33f88ce
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-05-03T18:02:51Z

    ZOOKEEPER-2174 JUnit4ZKTestRunner logs test failure for all exceptions
    
    JUnit4ZKTestRunner logs test failure for all exceptions, even if the test
    method is annotated with an expected exception (Chris Nauroth via rgs).
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1677463 13f79535-47bb-0310-9956-ffa450edef68

commit c7830c532e6a8fe0d14c82f1ad1a3ebcdfc83187
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-05-15T04:06:54Z

    ZOOKEEPER-2126 Improve exit log messsage of EventThread and SendThread by adding SessionId
(surendra singh lilhore via rakeshr)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1679491 13f79535-47bb-0310-9956-ffa450edef68

commit 1169a5b5eca3982d24359c27f669af429ff71157
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-05-18T07:38:36Z

    ZOOKEEPER-1077: C client lib doesn't build on Solaris (Chris Nauroth via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1679953 13f79535-47bb-0310-9956-ffa450edef68

commit 6b8f0cd84e22494a664fcb7369d15286b083a1ed
Author: Michi Mutsuzaki <michim@...>
Date:   2015-05-24T06:33:43Z

    ZOOKEEPER-2186 QuorumCnxManager#receiveConnection may crash with random input (rgs via
michim)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1681417 13f79535-47bb-0310-9956-ffa450edef68

commit ba96bd8a68c8eacab3a9e61b401cab0f35c7c314
Author: Michi Mutsuzaki <michim@...>
Date:   2015-05-24T06:37:32Z

    ZOOKEEPER-2124 Allow Zookeeper version string to have underscore '_' (Chris Nauroth via
michim)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1681418 13f79535-47bb-0310-9956-ffa450edef68

commit 6ce171a99878f745c0b7f212e27c9064f95cc4e2
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-05-29T19:51:58Z

    ZOOKEEPER-2179: Typo in Watcher.java (Archana T via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1682541 13f79535-47bb-0310-9956-ffa450edef68

commit e269cd5f4f76e8e2472de47354525674ad434659
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-02T20:43:29Z

    ZOOKEEPER-2096: C client builds with incorrect error codes in VisualStudio 2010+
    (Vitaly Stakhovsky via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683180 13f79535-47bb-0310-9956-ffa450edef68

commit a351209b3bd7e3123ce742789063784f6322d7fc
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-04T16:27:19Z

    ZOOKEEPER-2194: Let DataNode.getChildren() return an unmodifiable view of its
    children set (Hitoshi Mitake via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683589 13f79535-47bb-0310-9956-ffa450edef68

commit 67cb77e74d0d8389a5d58781d7ed9b44ca3b3c38
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-05T18:44:25Z

    ZOOKEEPER-2205: Log type of unexpected quorum packet in learner handler loop
    (Hitoshi Mitake via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683838 13f79535-47bb-0310-9956-ffa450edef68

commit 67019674379e6fac647d5eb23b694f1172d2849f
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-06T16:53:10Z

    ZOOKEEPER-2201: Network issues can cause cluster to hang due to near-deadlock
    (Donny Nadolny via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683931 13f79535-47bb-0310-9956-ffa450edef68

commit 6fdb35ff1e1d3fbd1f5f22f9d4a8581ef8244851
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-11T18:14:21Z

    ZOOKEEPER-2213: Empty path in Set crashes server and prevents restart
    (Hongchao Deng via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1684958 13f79535-47bb-0310-9956-ffa450edef68

commit 62888591c33968c3bf4fc79bb843cae9ff3ae5cf
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-13T00:49:48Z

    ZOOKEEPER-706: Large numbers of watches can cause session re-establishment to fail
    (Chris Thunes via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1685205 13f79535-47bb-0310-9956-ffa450edef68

commit b407b1593ec6d1b3879f532a3aff700cec50bf75
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-06-18T18:58:48Z

    ZOOKEEPER-602: log all exceptions not caught by ZK threads
    (Rakesh R via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1686289 13f79535-47bb-0310-9956-ffa450edef68

commit d66048c73a550aae8d34ccbae6ec384f3e8f74a3
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-07-06T15:48:24Z

    ZOOKEEPER-2224: Four letter command hangs when network is slow
    (Arshad Mohammad via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1689431 13f79535-47bb-0310-9956-ffa450edef68

commit 0d254c8df43acae08c71edd299bda35b593d2f06
Author: Michi Mutsuzaki <michim@...>
Date:   2015-07-28T04:39:20Z

    ZOOKEEPER-2235: License update (fpj via michim)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1692991 13f79535-47bb-0310-9956-ffa450edef68

commit b2703be61c7d6e31ec2b535f5f130b1703eefad9
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-08-01T03:39:21Z

    ZOOKEEPER-2237 Port async multi to 3.4 branch (Ivan Kelly via rakeshr)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1693680 13f79535-47bb-0310-9956-ffa450edef68

commit 8de5ad8d99a0f974a4393cc4eda546c66643dd96
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-08-05T20:17:18Z

    ZOOKEEPER-2239: JMX State from LocalPeerBean incorrect
    (Kevin Lee via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1694318 13f79535-47bb-0310-9956-ffa450edef68

commit cfff3f06ae5c208ab566ab4fd941df835b880e53
Author: Hongchao Deng <hdeng@...>
Date:   2015-08-17T20:52:07Z

    ZOOKEEPER-1907 Improve Thread handling (Rakesh R via hdeng)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1696337 13f79535-47bb-0310-9956-ffa450edef68

commit 908ea4084ea7b38b74b1fcaab7bfdf108d2a6093
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-08-25T05:13:25Z

    ZOOKEEPER-1927: zkServer.sh fails to read dataDir (and others)
    from zoo.cfg on Solaris 10 (grep issue, manifests as FAILED TO WRITE PID)
    (Chris Nauroth via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1697552 13f79535-47bb-0310-9956-ffa450edef68

commit 3a7c6a5f010086a2305bbbde5efbf6ddb2e767ed
Author: Flavio Paiva Junqueira <fpj@...>
Date:   2015-09-03T23:00:20Z

    ZOOKEEPER-2033: zookeeper follower fails to start after a restart immediately following
a new epoch (Asad Saeed via fpj)
    
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1701146 13f79535-47bb-0310-9956-ffa450edef68

commit 1c1a2c785b4b18e639ff4497d96d721aaf11eb40
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-09-06T17:48:13Z

    ZOOKEEPER-2256: Zookeeper is not using specified JMX port in zkEnv.sh(Arshad Mohammad
via rakeshr)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1701503 13f79535-47bb-0310-9956-ffa450edef68

commit 8f326d5f5b2af8710ec64941fa5db7bca6bc06da
Author: Raúl Gutiérrez Segalés <rgs@...>
Date:   2015-09-10T04:12:05Z

    ZOOKEEPER-2270: Allow MBeanRegistry to be overridden for better unit tests
    (Jordan Zimmerman via rgs)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1702157 13f79535-47bb-0310-9956-ffa450edef68

commit 124670eb739d064e6bd01da451e92fe0084bdc6c
Author: Chris Nauroth <cnauroth@...>
Date:   2015-09-11T06:02:29Z

    ZOOKEEPER-2040: Server to log underlying cause of SASL connection problems. (Steve Loughran
via cnauroth)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1702381 13f79535-47bb-0310-9956-ffa450edef68

commit 9350096d66c4e405f312a453185471b2306fb9ee
Author: Rakesh Radhakrishnan <rakeshr@...>
Date:   2015-09-17T07:12:00Z

    ZOOKEEPER-2245: SimpleSysTest test cases fails (Arshad Mohammad via rakeshr)
    
    git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1703504 13f79535-47bb-0310-9956-ffa450edef68

----


---

Mime
View raw message