Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Received-SPF: pass (athena.apache.org: domain of mathias.hodler@gmail.com
 designates 209.85.215.181 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAKR25EQypcf5Cm=9i4CYOJpTXbSNus1t8cijoZXbdLcZ09mwjA@mail.gmail.com>
References: 
 <CAMdLfTWOgz1-FbUn-=V+HUg4WjjcMD-WDaaR=n3VJtVhqwfiqA@mail.gmail.com>
	<CAKR25EQypcf5Cm=9i4CYOJpTXbSNus1t8cijoZXbdLcZ09mwjA@mail.gmail.com>
Date: Mon, 8 Apr 2013 18:40:08 +0200
Message-ID: 
 <CAMdLfTVn+EBahSD_+COiy4Ok9coXqhayWWGmzN96ZYy52Cs59w@mail.gmail.com>
Subject: Re: Zookeeper does not clean up deleted nodes
From: Mathias Hodler <mathias.hodler@gmail.com>
To: user@zookeeper.apache.org
Content-Type: multipart/alternative; boundary=001a11c2899e052aa104d9dc1791

--001a11c2899e052aa104d9dc1791
Content-Type: text/plain; charset=ISO-8859-1

Thanks, this could be the reason. I only used a single zookeeper server, so
it should act as a leader.

So if I need to store larger files (about 1MB) the only option is to
increase the heap space? I know that zookeeper is designed for small files,
but I'm using zookeeper with solr and solr stores all the index
configuration with large dictionaries in zookeeper.


2013/4/8 Benjamin Reed <breed@apache.org>

> are you looking at the leader or the follower? the leader keeps the last
> few transactions in memory to speed up syncing with new followers. that
> might be what you are seeing.
>
>
> On Mon, Apr 8, 2013 at 3:32 AM, Mathias Hodler <mathias.hodler@gmail.com
> >wrote:
>
> > Hi,
> >
> > I made some tests and it seems like zookeeper doesn't clean up the last
> 500
> > deleted nodes.
> >
> > In my test I created nodes and deleted each node after it was created. I
> > repeated this step 1000 times and then triggered a full gc. These are the
> > results
> >
> > Creating 1000 Nodes and deleting 1000 Nodes and each node has...
> > ...1000kb data = 529MB heap used after FullGC
> > ...500kb data = 281MB heap used after FullGC
> > ...256kb data = 140MB heap used after FullGC
> > ...128kb data =  68MB heap used after FullGC
> >
> > If I'm creating 1000 nodes with each 1000kb data and then deleting the
> > nodes and after that creating 1000 nodes with 128kb data and deleting the
> > nodes again, 68MB heap space is used.
> >
> > So it seems Zookeeper caches / doesn't clean up the last 500 deleted
> nodes.
> >
> > Is this a bug or are there configuration parameter to change that
> > behaviour?
> >
>

--001a11c2899e052aa104d9dc1791--