zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lvfangmin <...@git.apache.org>
Subject [GitHub] zookeeper pull request #590: [ZOOKEEPER-1177] Add the memory optimized watch...
Date Mon, 06 Aug 2018 23:51:30 GMT
GitHub user lvfangmin opened a pull request:

    https://github.com/apache/zookeeper/pull/590

    [ZOOKEEPER-1177] Add the memory optimized watch manager for concentrate watches scenario

    The current HashSet based WatcherManager will consume more than 40GB memory when 
    creating 300M watches.
    
    This patch optimized the memory and time complexity for concentrate watches scenario,
compared to WatchManager, both the memory consumption and time complexity improved a lot.
I'll post more data later with micro benchmark result.
    
    Changed made compared to WatchManager:
    * Only keep path to watches map
    * Use BitSet to save the memory used to store watches
    * Use ConcurrentHashMap and ReadWriteLock instead of synchronized to reduce lock retention
    * Lazily clean up the closed watchers

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lvfangmin/zookeeper ZOOKEEPER-1177

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zookeeper/pull/590.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #590
    
----
commit d4f996fdd760417c90ffb28fd63cc37dc87416c1
Author: Fangmin Lyu <allenlyu@...>
Date:   2018-08-06T21:43:22Z

    add the memory optimized watch manager for concentrate watches

----


---

Mime
View raw message