zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Weisberg <aweisb...@voltdb.com>
Subject Re: ZooKeeper Memory Usage
Date Thu, 09 Feb 2012 22:25:56 GMT
Hi,

If you set -Xmx2g or so and also run with -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp you will get a heap dump. If you run (
http://www.eclipse.org/mat/) on it then ye olde pack rat will stick out
like a sore thumb.

Ariel

On Thu, Feb 9, 2012 at 1:17 PM, Mahadev Konar <mahadev@hortonworks.com>wrote:

> This is interesting and important.
>
> Cesar, what jvm options are you running with? Can you the options in:
>
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
>
> Atleast get the GC logs that we can look at?
>
> This will be very interesting.
>
> mahadev
>
>
> 2012/2/9 César Álvarez Núñez <cesar.alvarez.nunez@gmail.com>:
> > In my case, our stress test show up a linear increase of "tenured memory"
> > from 0 to > 3GiB with ZK 3.4.0 whereas the same stress-test with 3.3.3
> > keeps "tenured memory" stable and < 10MiB.
> >
> > The stress test performs many zNodes creation and delete but the overall
> zk
> > usage at any moment in time was relative small.
> >
> > BR,
> > /César.
> >
> > On Thu, Feb 9, 2012 at 3:14 PM, Camille Fournier <camille@apache.org>
> wrote:
> >
> >> This is really a question about how the jvm grows its heaps and resizes
> >> them. If the jvm cannot allocate enough memory for the process because
> you
> >> didn't set the max memory high enough, it will fall over. Zookeeper
> keeps
> >> its entire state in memory for performance reasons, if it were to swap
> that
> >> would be quite bad for performance.
> >>
> >> C
> >> On Feb 8, 2012 8:23 PM, "Mike Schilli" <m@perlmeister.com> wrote:
> >>
> >> > We've got a ZooKeeper instance that's using about 5 GB of resident
> >> > memory. Every time we restart it, it starts at 200MB, and then grows
> >> > slowly until it is back at 5 GB.
> >> >
> >> > The large footprint is related to how much data we've got in there.
> >> > What's interesting, though, is that the process size doesn't shrink if
> >> > we purge some of the data.
> >> >
> >> > Now, this isn't a big problem, I'm just curious if the process will
> fall
> >> > over at some point if it can't get more memory or if it'll just make
> due
> >> > by caching less data.
> >> >
> >> > Also, if I remember correctly, there's a configuration variable to set
> >> > the maximum size, what happens if ZK reaches that?
> >> >
> >> > -- -- Mike
> >> >
> >> > Mike Schilli
> >> > m@perlmeister.com
> >> >
> >>
>
>
>
> --
> Mahadev Konar
> Hortonworks Inc.
> http://hortonworks.com/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message