curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Zimmerman <jor...@jordanzimmerman.com>
Subject Re: InterProcessSemaphoreMutex not cleaning up old znodes
Date Mon, 14 Sep 2015 13:47:40 GMT
ChildReaper uses a Reaper internally. So, just use whichever one works for you.

-Jordan



On September 14, 2015 at 8:30:19 AM, Jens Rantil (jens.rantil@tink.se) wrote:

Hi again,

A small follow-up question. What are the pros and cons of using Reaper vs. ChildReaper? It
seems to me that using ChildReaper will involve a lot less intrusive code. Are there performance
implications of one or the other?

Thanks,
Jens

On Sun, Sep 13, 2015 at 10:27 PM, Jordan Zimmerman <jordan@jordanzimmerman.com> wrote:
This has long been an issue in ZooKeeper. The good news is that there are solutions:

* Current releases: Curator has utilities for cleaning up old parent nodes. Look at Reaper
and ChildReaper
* New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) support the new Container
node feature. Curator 2.9.0 will automatically create container nodes which are automatically
cleaned up when empty.

-Jordan


On September 13, 2015 at 11:41:41 AM, Jens Rantil (jens.rantil@tink.se) wrote:

Hi again,

I have a distributed application which has InterProcessSemaphoreMutexes which occasionally
are taken per user. To do this, the mutex path is appended with a user's unique ID (example: /mylock/{userid}).
All is fine and dandy and this generally works. However, I've noticed that the paths given
to the InterProcessSemaphoreMutexes aren't cleaned up after I'm done with the locks.

This means, my Zookeeper ensemble has every single one of my previous mutexes held in memory.
When I list my znodes I see:
/mylock/1
/mylock/2
...
/mylock/XXX

Given that I generally only take ~20 locks at a time it feels unnecessary to keep all these
in memory. We have had to increase initLimit in Zookeeper configuration, most certainly due
to all of these znodes.

Two questions:
Is there a reason for keeping these znodes hanging around?
Given that I need to use mutexes, what are the best practise for taking locks per user like
this? Hash the user IDs and bucket them into a limitted number of mutexes (enough to avoid
contention)? Have a separate cleaning thread that cleans up old znodes? How are you guys handling
this?
Thanks,
Jens

--
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter



--
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter
Mime
View raw message