Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Received-SPF: pass (athena.apache.org: domain of neha.narkhede@gmail.com
 designates 209.85.212.42 as permitted sender)
Received-SPF: pass (google.com: domain of neha.narkhede@gmail.com designates
 10.52.17.239 as permitted sender) client-ip=10.52.17.239;
MIME-Version: 1.0
In-Reply-To: 
 <CALF7aMTHpy+-7DPPE-+UTeUQRGYqW3vFxtP0sb4vducoK-510g@mail.gmail.com>
References: <alpine.LRH.2.02.1202081713090.7232@pyramidlake.corp.yahoo.com>
	<CAKF1A=vmvGAeCpHp9Nha+hYjJUiQEgG8LJ6yCgEgiE9zKPKxNg@mail.gmail.com>
	<CAFcPz7h0W0qtmjMZYe034mJza0S6NebvGG5TfgYSO3Km6ZZDDA@mail.gmail.com>
	<CALF7aMSHzUMACmXf6UrRvB5sFXC9aM02uRTo984NefMJoF4oQw@mail.gmail.com>
	<CAFcPz7j5_nxz2ryRw_PcDx5E6wrrWNRhtdesBLPtjHOtweHkUA@mail.gmail.com>
	<CALF7aMTHpy+-7DPPE-+UTeUQRGYqW3vFxtP0sb4vducoK-510g@mail.gmail.com>
Date: Fri, 2 Mar 2012 15:18:05 -0800
Message-ID: 
 <CAOG_4QZSN1=qs+w_NcRYpdLinVmvty6ui-Urs64YigiECDyeSQ@mail.gmail.com>
Subject: Re: ZooKeeper Memory Usage
From: Neha Narkhede <neha.narkhede@gmail.com>
To: user@zookeeper.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Was there ever a JIRA created for this issue ?

Thanks,
Neha

On Fri, Feb 10, 2012 at 9:31 AM, Mahadev Konar <mahadev@hortonworks.com> wr=
ote:
> Great. You should have the gc logs then. Mind creating a jira and
> uploading to it?
>
> mahadev
>
> 2012/2/10 C=E9sar =C1lvarez N=FA=F1ez <cesar.alvarez.nunez@gmail.com>:
>> This is the java.env file content.
>>
>> now=3D`date +%d%m%Y_%H%M%S`
>>
>> gcLogFile=3D"/srv/zk/GC/`hostname`-${now}.log"
>> gcOpts=3D"${gcOpts} -Xloggc:$gcLogFile"
>> gcOpts=3D"${gcOpts} -XX:+PrintGC"
>> gcOpts=3D"${gcOpts} -XX:+PrintGCTimeStamps"
>> gcOpts=3D"${gcOpts} -XX:+PrintGCDetails"
>> gcOpts=3D"${gcOpts} -XX:+PrintTenuringDistribution"
>> gcOpts=3D"${gcOpts} -XX:+PrintHeapAtGC"
>>
>> gcOpts=3D"${gcOpts} -XX:+AggressiveHeap"
>> #https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
>> #gcOpts=3D"${gcOpts} -XX:+UseConcMarkSweepGC"
>> #gcOpts=3D"${gcOpts} -XX:ParallelGCThreads=3D8"
>> #gcOpts=3D"-Xms128M -Xmx1G"
>>
>> jvmOpts=3D"${jvmOpts} -d64"
>> jvmOpts=3D"${jvmOpts} -server"
>> jvmOpts=3D"${jvmOpts} -XX:+UseCompressedOops"
>>
>> JVMFLAGS=3D"${gcOpts} ${jvmOpts}"
>>
>> /C=E9sar.
>>
>> On Thu, Feb 9, 2012 at 7:17 PM, Mahadev Konar <mahadev@hortonworks.com>w=
rote:
>>
>>> This is interesting and important.
>>>
>>> Cesar, what jvm options are you running with? Can you the options in:
>>>
>>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
>>>
>>> Atleast get the GC logs that we can look at?
>>>
>>> This will be very interesting.
>>>
>>> mahadev
>>>
>>>
>>> 2012/2/9 C=E9sar =C1lvarez N=FA=F1ez <cesar.alvarez.nunez@gmail.com>:
>>> > In my case, our stress test show up a linear increase of "tenured mem=
ory"
>>> > from 0 to > 3GiB with ZK 3.4.0 whereas the same stress-test with 3.3.=
3
>>> > keeps "tenured memory" stable and < 10MiB.
>>> >
>>> > The stress test performs many zNodes creation and delete but the over=
all
>>> zk
>>> > usage at any moment in time was relative small.
>>> >
>>> > BR,
>>> > /C=E9sar.
>>> >
>>> > On Thu, Feb 9, 2012 at 3:14 PM, Camille Fournier <camille@apache.org>
>>> wrote:
>>> >
>>> >> This is really a question about how the jvm grows its heaps and resi=
zes
>>> >> them. If the jvm cannot allocate enough memory for the process becau=
se
>>> you
>>> >> didn't set the max memory high enough, it will fall over. Zookeeper
>>> keeps
>>> >> its entire state in memory for performance reasons, if it were to sw=
ap
>>> that
>>> >> would be quite bad for performance.
>>> >>
>>> >> C
>>> >> On Feb 8, 2012 8:23 PM, "Mike Schilli" <m@perlmeister.com> wrote:
>>> >>
>>> >> > We've got a ZooKeeper instance that's using about 5 GB of resident
>>> >> > memory. Every time we restart it, it starts at 200MB, and then gro=
ws
>>> >> > slowly until it is back at 5 GB.
>>> >> >
>>> >> > The large footprint is related to how much data we've got in there=
.
>>> >> > What's interesting, though, is that the process size doesn't shrin=
k if
>>> >> > we purge some of the data.
>>> >> >
>>> >> > Now, this isn't a big problem, I'm just curious if the process wil=
l
>>> fall
>>> >> > over at some point if it can't get more memory or if it'll just ma=
ke
>>> due
>>> >> > by caching less data.
>>> >> >
>>> >> > Also, if I remember correctly, there's a configuration variable to=
 set
>>> >> > the maximum size, what happens if ZK reaches that?
>>> >> >
>>> >> > -- -- Mike
>>> >> >
>>> >> > Mike Schilli
>>> >> > m@perlmeister.com
>>> >> >
>>> >>
>>>
>>>
>>>
>>> --
>>> Mahadev Konar
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>
>
>
> --
> Mahadev Konar
> Hortonworks Inc.
> http://hortonworks.com/