Mailing-List: contact user-help@curator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@curator.apache.org
MIME-Version: 1.0
From: Jens Rantil <jens.rantil@tink.se>
Date: Sun, 13 Sep 2015 18:41:04 +0200
Message-ID: 
 <CAL+ArfWZ6TsY1GqMPAT-rqn9cTiXbz1qnmz9SHywTP6qs=un+g@mail.gmail.com>
Subject: InterProcessSemaphoreMutex not cleaning up old znodes
To: user@curator.apache.org
Content-Type: multipart/alternative; boundary=94eb2c05509a8925ed051fa39e06

--94eb2c05509a8925ed051fa39e06
Content-Type: text/plain; charset=UTF-8

Hi again,

I have a distributed application which has InterProcessSemaphoreMutexes which
occasionally are taken per user. To do this, the mutex path is appended
with a user's unique ID (example: /mylock/{userid}). All is fine and dandy
and this generally works. However, I've noticed that the paths given to the
InterProcessSemaphoreMutexes aren't cleaned up after I'm done with the
locks.

This means, my Zookeeper ensemble has every single one of my previous
mutexes held in memory. When I list my znodes I see:
/mylock/1
/mylock/2
...
/mylock/XXX

Given that I generally only take ~20 locks at a time it feels unnecessary
to keep all these in memory. We have had to increase initLimit in Zookeeper
configuration, most certainly due to all of these znodes.

Two questions:

   - Is there a reason for keeping these znodes hanging around?
   - Given that I need to use mutexes, what are the best practise for
   taking locks per user like this? Hash the user IDs and bucket them into a
   limitted number of mutexes (enough to avoid contention)? Have a separate
   cleaning thread that cleans up old znodes? How are you guys handling this?

Thanks,
Jens

-- 
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook <https://www.facebook.com/#!/tink.se> Linkedin
<http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
 Twitter <https://twitter.com/tink>

--94eb2c05509a8925ed051fa39e06
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi again,<div><br></div><div>I have a distributed applicat=
ion which has=C2=A0<font face=3D"monospace, monospace">InterProcessSemaphor=
eMutex</font><font face=3D"arial, helvetica, sans-serif">es</font>=C2=A0whi=
ch occasionally are taken per user. To do this, the mutex path is appended =
with a user&#39;s unique ID (example:=C2=A0<font face=3D"monospace, monospa=
ce">/mylock/{userid}</font>). All is fine and dandy and this generally work=
s. However, I&#39;ve noticed that the paths given to the=C2=A0<span style=
=3D"font-family:monospace,monospace">InterProcessSemaphoreMutex</span><font=
 face=3D"arial, helvetica, sans-serif">es aren&#39;t cleaned up after I&#39=
;m done with the locks.</font></div><div><font face=3D"arial, helvetica, sa=
ns-serif"><br></font></div><div><font face=3D"arial, helvetica, sans-serif"=
>This means, my Zookeeper ensemble has every single one of my previous mute=
xes held in memory. When I list my znodes I see:</font></div><div><span sty=
le=3D"font-family:monospace,monospace">/mylock/1</span></div><div><span sty=
le=3D"font-family:monospace,monospace">/mylock/2</span></div><div><font fac=
e=3D"monospace, monospace">...</font></div><div><span style=3D"font-family:=
monospace,monospace">/mylock/XXX</span><font face=3D"monospace, monospace">=
<br></font><div><br></div><div>Given that I generally only take ~20 locks a=
t a time it feels unnecessary to keep all these in memory. We have had to i=
ncrease=C2=A0<span style=3D"font-family:monospace,monospace;font-size:12.8p=
x">initLimit</span><span style=3D"font-size:12.8px"><font face=3D"arial, he=
lvetica, sans-serif"> in Zookeeper configuration, most certainly due to all=
 of these znodes.</font></span><br></div><div><br></div><div>Two questions:=
</div><div><ul><li>Is there a reason for keeping these znodes hanging aroun=
d?</li><li>Given that I need to use mutexes, what are the best practise for=
 taking locks per user like this? Hash the user IDs and bucket them into a =
limitted number of mutexes (enough to avoid contention)? Have a separate cl=
eaning thread that cleans up old znodes? How are you guys handling this?<br=
></li></ul><div>Thanks,</div></div><div>Jens</div><div><br></div>-- <br><di=
v class=3D"gmail_signature"><div dir=3D"ltr"><div>Jens Rantil</div><div>Bac=
kend engineer</div><div>Tink AB</div><div><br></div><div>Email:=C2=A0<a hre=
f=3D"mailto:jens.rantil@tink.se" style=3D"color:rgb(17,85,204)" target=3D"_=
blank">jens.rantil@tink.se</a></div><div>Phone: +46 708 84 18 32</div><div>=
Web:=C2=A0<a href=3D"http://www.tink.se/" style=3D"color:rgb(17,85,204)" ta=
rget=3D"_blank">www.tink.se</a></div><div><br></div><div><a href=3D"https:/=
/www.facebook.com/#!/tink.se" style=3D"color:rgb(17,85,204);font-family:ari=
al;font-size:small" target=3D"_blank">Facebook</a><span style=3D"font-famil=
y:arial;font-size:small">=C2=A0</span><a href=3D"http://www.linkedin.com/co=
mpany/2735919?trk=3Dvsrp_companies_res_photo&amp;trkInfo=3DVSRPsearchId%3A1=
057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary" style=
=3D"color:rgb(17,85,204);font-family:arial;font-size:small" target=3D"_blan=
k">Linkedin</a><span style=3D"font-family:arial;font-size:small">=C2=A0</sp=
an><a href=3D"https://twitter.com/tink" style=3D"color:rgb(17,85,204);font-=
family:arial;font-size:small" target=3D"_blank">Twitter</a></div></div></di=
v>
</div></div>

--94eb2c05509a8925ed051fa39e06--