curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Rantil <jens.ran...@tink.se>
Subject Re: InterProcessSemaphoreMutex not cleaning up old znodes
Date Mon, 14 Sep 2015 10:01:37 GMT
Thanks, Jordan. The ChildReaper is exactly what I was looking for.

I'll hold off regarding upgrading of Zookeeper and Curator for now.

Cheers,
Jens

On Sun, Sep 13, 2015 at 10:27 PM, Jordan Zimmerman <
jordan@jordanzimmerman.com> wrote:

> This has long been an issue in ZooKeeper. The good news is that there are
> solutions:
>
> * Current releases: Curator has utilities for cleaning up old parent
> nodes. Look at Reaper and ChildReaper
> * New releases: ZooKeeper 3.5.1 + Curator 2.9.0 (just released) support
> the new Container node feature. Curator 2.9.0 will automatically create
> container nodes which are automatically cleaned up when empty.
>
> -Jordan
>
>
> On September 13, 2015 at 11:41:41 AM, Jens Rantil (jens.rantil@tink.se)
> wrote:
>
> Hi again,
>
> I have a distributed application which has InterProcessSemaphoreMutexes which
> occasionally are taken per user. To do this, the mutex path is appended
> with a user's unique ID (example: /mylock/{userid}). All is fine and
> dandy and this generally works. However, I've noticed that the paths given
> to the InterProcessSemaphoreMutexes aren't cleaned up after I'm done with
> the locks.
>
> This means, my Zookeeper ensemble has every single one of my previous
> mutexes held in memory. When I list my znodes I see:
> /mylock/1
> /mylock/2
> ...
> /mylock/XXX
>
> Given that I generally only take ~20 locks at a time it feels unnecessary
> to keep all these in memory. We have had to increase initLimit in
> Zookeeper configuration, most certainly due to all of these znodes.
>
> Two questions:
>
>    - Is there a reason for keeping these znodes hanging around?
>    - Given that I need to use mutexes, what are the best practise for
>    taking locks per user like this? Hash the user IDs and bucket them into a
>    limitted number of mutexes (enough to avoid contention)? Have a separate
>    cleaning thread that cleans up old znodes? How are you guys handling this?
>
> Thanks,
> Jens
>
> --
> Jens Rantil
> Backend engineer
> Tink AB
>
> Email: jens.rantil@tink.se
> Phone: +46 708 84 18 32
> Web: www.tink.se
>
> Facebook <https://www.facebook.com/#!/tink.se> Linkedin
> <http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
>  Twitter <https://twitter.com/tink>
>
>


-- 
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook <https://www.facebook.com/#!/tink.se> Linkedin
<http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
 Twitter <https://twitter.com/tink>

Mime
View raw message