From dev-return-72130-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Tue Aug 7 01:51:32 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 8C32F180627 for ; Tue, 7 Aug 2018 01:51:32 +0200 (CEST) Received: (qmail 99729 invoked by uid 500); 6 Aug 2018 23:51:31 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Received: (qmail 99717 invoked by uid 99); 6 Aug 2018 23:51:31 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Aug 2018 23:51:30 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id DB7DBDFC7C; Mon, 6 Aug 2018 23:51:30 +0000 (UTC) From: lvfangmin To: dev@zookeeper.apache.org Reply-To: dev@zookeeper.apache.org Message-ID: Subject: [GitHub] zookeeper pull request #590: [ZOOKEEPER-1177] Add the memory optimized watch... Content-Type: text/plain Date: Mon, 6 Aug 2018 23:51:30 +0000 (UTC) GitHub user lvfangmin opened a pull request: https://github.com/apache/zookeeper/pull/590 [ZOOKEEPER-1177] Add the memory optimized watch manager for concentrate watches scenario The current HashSet based WatcherManager will consume more than 40GB memory when creating 300M watches. This patch optimized the memory and time complexity for concentrate watches scenario, compared to WatchManager, both the memory consumption and time complexity improved a lot. I'll post more data later with micro benchmark result. Changed made compared to WatchManager: * Only keep path to watches map * Use BitSet to save the memory used to store watches * Use ConcurrentHashMap and ReadWriteLock instead of synchronized to reduce lock retention * Lazily clean up the closed watchers You can merge this pull request into a Git repository by running: $ git pull https://github.com/lvfangmin/zookeeper ZOOKEEPER-1177 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/590.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #590 ---- commit d4f996fdd760417c90ffb28fd63cc37dc87416c1 Author: Fangmin Lyu Date: 2018-08-06T21:43:22Z add the memory optimized watch manager for concentrate watches ---- ---