zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lvfangmin <...@git.apache.org>
Subject [GitHub] zookeeper issue #590: [ZOOKEEPER-1177] Add the memory optimized watch manage...
Date Fri, 31 Aug 2018 23:36:30 GMT
Github user lvfangmin commented on the issue:

    https://github.com/apache/zookeeper/pull/590
  
    Added JMH micro benchmark for the watch manager:
    
    * It shows **big win** for the watch heavy cases, with the current implementation, it
uses more than 50MB memory to store 1M watches, with WatchManagerOptimized it only uses around
0.2MB. 
    * It also makes add and trigger watches more efficient, since WatchManagerOptimized doesn't
maintain the reverse map. 
    * In sparse watches use case, the WatchManagerOptimized is expected to use a bit more
memory because it needs extra effort to maintain those bit set. In the test it shows around
10% more memory usage.
    
    Here are more result about the throughput/latency related with WatchManager: 
    
    ```
    Benchmark                               (pathCount)    (watchManagerClass)  (watcherCount)
 Mode  Cnt   Score    Error  Units
    WatchBench.testAddConcentrateWatch            10000           WatchManager           
 N/A  avgt    9   5.382 ±  0.968  ms/op
    WatchBench.testAddConcentrateWatch            10000  WatchManagerOptimized           
 N/A  avgt    9   0.696 ±  0.133  ms/op
    WatchBench.testAddSparseWatch                 10000           WatchManager           10000
 avgt    9   4.889 ±  1.585  ms/op
    WatchBench.testAddSparseWatch                 10000  WatchManagerOptimized           10000
 avgt    9   4.794 ±  1.068  ms/op
    WatchBench.testTriggerConcentrateWatch            1           WatchManager           
   1  avgt    9  ≈ 10⁻⁴           ms/op
    WatchBench.testTriggerConcentrateWatch            1           WatchManager           
1000  avgt    9   0.037 ±  0.002  ms/op
    WatchBench.testTriggerConcentrateWatch            1  WatchManagerOptimized           
   1  avgt    9  ≈ 10⁻⁴           ms/op
    WatchBench.testTriggerConcentrateWatch            1  WatchManagerOptimized           
1000  avgt    9   0.025 ±  0.001  ms/op
    WatchBench.testTriggerConcentrateWatch         1000           WatchManager           
   1  avgt    9   0.048 ±  0.003  ms/op
    WatchBench.testTriggerConcentrateWatch         1000           WatchManager           
1000  avgt    9  71.838 ±  4.043  ms/op
    WatchBench.testTriggerConcentrateWatch         1000  WatchManagerOptimized           
   1  avgt    9   0.079 ±  0.002  ms/op
    WatchBench.testTriggerConcentrateWatch         1000  WatchManagerOptimized           
1000  avgt    9  26.135 ±  0.223  ms/op
    WatchBench.testTriggerSparseWatch             10000           WatchManager           10000
 avgt    9   1.207 ±  0.035  ms/op
    WatchBench.testTriggerSparseWatch             10000  WatchManagerOptimized           10000
 avgt    9   1.321 ±  0.019  ms/op
    ```
    
    You can try the following command to run the micro benchmark:
    ```
    $ ant clean package
    $ ant clean package -buildfile zookeeper-contrib/zookeeper-contrib-fatjar/build.xml
    $ java -jar build/contrib/fatjar/zookeeper-dev-fatjar.jar jmh
    ```
    
    @maoling @anmolnar hope this gives you a more vivid comparison between the old and new
watch manager implementation.


---

Mime
View raw message