zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Rantil <jens.ran...@tink.se>
Subject Re: Cleaning up giant znode
Date Sat, 27 Aug 2016 10:12:42 GMT
Hi,

Since we had so many I znodes I was afraid of doing a rolling restart of
the cluster. We generally have a lot of znodes and I've had issues with
rolling restarts (container znodes will fix this for us).

My woraround was this: I installed binutils' `strings` command on the
server and extracted the znode names from the snapshot file. I then
iterated over those names to delete the znodes in question. Worked like a
charm and avoided a restart.

Thanks for your input. Let's hope this never happens again :-)

Cheers,
Jens

On Thursday, August 25, 2016, Dan Benediktson
<dbenediktson@twitter.com.invalid> wrote:

> Yes, IIRC, the packet limit is enforced differently on server side and
> client side: the server side imposes it per-node that it sends, while the
> client imposes it for the whole response message. Since your problem is not
> a single big node, but a long list of nodes, I expect if you just override
> jute.maxbuffer on the client side to a big enough value (using the system
> property when launching your client), you should be able to list the nodes
> successfully.
>
> Dan
>
> On Thu, Aug 25, 2016 at 1:19 PM, Galo Navarro <anglorvaroa@gmail.com
> <javascript:;>> wrote:
>
> > Have you tried raising jute.maxbuffer [1]? That might work around the
> > listing issue and let you delete children individually.
> >
> > [1]: https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#
> > Experimental+Options%2FFeatures
> >
> > Cheers,
> > Galo
> >
> > On 25 August 2016 at 21:51, Jens Rantil <jens.rantil@tink.se
> <javascript:;>> wrote:
> >
> > > Hi,
> > >
> > > A code mistake lead to us to write a lot of znodes with very random
> names
> > > under the same directory/hierarchy in our Zookeeper ensemble.
> > > Unfortunately, I am reaching the packet limit when trying to list the
> > > nodes, so I can't delete them.
> > >
> > > Does anyone have any idea how I could clean up these nodes?
> > >
> > > Thanks,
> > > Jens
> > > --
> > >
> > > Jens Rantil
> > > Backend Developer @ Tink
> > >
> > > Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden
> > > For urgent matters you can reach me at +46-708-84 18 32.
> > >
> >
>


-- 
Jens Rantil
Backend engineer
Tink AB

Email: jens.rantil@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook <https://www.facebook.com/#!/tink.se> Linkedin
<http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_photo&trkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary>
 Twitter <https://twitter.com/tink>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message