Return-Path: X-Original-To: apmail-curator-user-archive@minotaur.apache.org Delivered-To: apmail-curator-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4025A18E98 for ; Sun, 13 Sep 2015 20:27:16 +0000 (UTC) Received: (qmail 36010 invoked by uid 500); 13 Sep 2015 20:27:16 -0000 Delivered-To: apmail-curator-user-archive@curator.apache.org Received: (qmail 35961 invoked by uid 500); 13 Sep 2015 20:27:16 -0000 Mailing-List: contact user-help@curator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@curator.apache.org Delivered-To: mailing list user@curator.apache.org Received: (qmail 35951 invoked by uid 99); 13 Sep 2015 20:27:16 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Sep 2015 20:27:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 8EC05C0041 for ; Sun, 13 Sep 2015 20:27:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.001 X-Spam-Level: *** X-Spam-Status: No, score=3.001 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 0F1F6qkkCrtR for ; Sun, 13 Sep 2015 20:27:04 +0000 (UTC) Received: from mail-ig0-f180.google.com (mail-ig0-f180.google.com [209.85.213.180]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 28AD120107 for ; Sun, 13 Sep 2015 20:27:04 +0000 (UTC) Received: by igcpb10 with SMTP id pb10so77149257igc.1 for ; Sun, 13 Sep 2015 13:27:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:content-type; bh=+KfhspxwUZ+gY8TNoQmCoPV527ggfH7zk2vo55b8ntE=; b=Hk5NwI/FUj7k8bWjhc9pffXTEONCPyV1K46wk2jo2abgTeFBc8GtkD+F4OcDYGyc4n DaBUgFawnWYE992M34LTLpzH8yabMHL22xOcQOnIvPjivEghmuAY04RyEfnzi7X3O4y9 dDwQFBZ9YDo8JVic+qUhr5lUcMzdJodqpyT6bpbcZC050TqloZ9mkt00Zm69zAsNSjsP 7UAFbXLTnMTl2m4nuWP7HEMRemjtwAvArebx+VvmYrn+uQ2vR3f1n4CzXE6wlDq0tvRE Dpvxnx22oCO5i9fFXNOXvXp5iL3ocy6Ars8foKh7Ln/MPo3Tspkd+aVzORSwi99k54l+ s1Yg== X-Gm-Message-State: ALoCoQk5x+5oGs/RVtOf+DsTcTZDITHMY9JYrbgyhJ37Dea3G/b+/2bQjO4nNZAje75F2/Yu110c X-Received: by 10.50.3.66 with SMTP id a2mr13199186iga.92.1442176023571; Sun, 13 Sep 2015 13:27:03 -0700 (PDT) Received: from Jordans-MacBook-Pro.local ([190.140.148.131]) by smtp.gmail.com with ESMTPSA id u99sm4746043ioi.0.2015.09.13.13.27.01 (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 13 Sep 2015 13:27:02 -0700 (PDT) Date: Sun, 13 Sep 2015 15:27:00 -0500 From: Jordan Zimmerman To: Jens Rantil , user@curator.apache.org Message-ID: In-Reply-To: References: Subject: Re: InterProcessSemaphoreMutex not cleaning up old znodes X-Mailer: Airmail (303) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="55f5dc14_722eab75_42a" --55f5dc14_722eab75_42a Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline This has long been an issue in ZooKeeper. The good news is that there are= solutions: * Current releases: Curator has utilities for cleaning up old parent node= s. Look at Reaper and ChildReaper * New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) support t= he new Container node feature. Curator 2.9.0 will automatically create co= ntainer nodes which are automatically cleaned up when empty. -Jordan On September 13, 2015 at 11:41:41 AM, Jens Rantil (jens.rantil=40tink.se)= wrote: Hi again, I have a distributed application which has=C2=A0InterProcessSemaphoreMute= xes=C2=A0which occasionally are taken per user. To do this, the mutex pat= h is appended with a user's unique ID (example:=C2=A0/mylock/=7Buserid=7D= ). All is fine and dandy and this generally works. However, I've noticed = that the paths given to the=C2=A0InterProcessSemaphoreMutexes aren't clea= ned up after I'm done with the locks. This means, my Zookeeper ensemble has every single one of my previous mut= exes held in memory. When I list my znodes I see: /mylock/1 /mylock/2 ... /mylock/XXX Given that I generally only take =7E20 locks at a time it feels unnecessa= ry to keep all these in memory. We have had to increase=C2=A0initLimit in= Zookeeper configuration, most certainly due to all of these znodes. Two questions: Is there a reason for keeping these znodes hanging around=3F Given that I need to use mutexes, what are the best practise for taking l= ocks per user like this=3F Hash the user IDs and bucket them into a limit= ted number of mutexes (enough to avoid contention)=3F Have a separate cle= aning thread that cleans up old znodes=3F How are you guys handling this=3F= Thanks, Jens -- Jens Rantil Backend engineer Tink AB Email:=C2=A0jens.rantil=40tink.se Phone: +46 708 84 18 32 Web:=C2=A0www.tink.se =46acebook=C2=A0Linkedin=C2=A0Twitter --55f5dc14_722eab75_42a Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline