zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: setting zookeeper heap size
Date Fri, 11 Nov 2011 01:41:06 GMT
Re wiseness - that all depends on the size of the content you are storing. ;-)

You could get more insight using jmap and the like to get an idea of
how much mem you are using. (in a test setup say)

Patrick

On Thu, Nov 10, 2011 at 5:32 PM, Ed Sexton <sextonhadoop@gmail.com> wrote:
> Thanks all, I am understanding things a bit better.
>
> Here is the output of the heap.  1/4 of the available memory (24gb) does
> put it at ~6gb.
>
> I will override using the JVMFLAG and test it out.  If this is not a wise
> thing to do, please let me know - thanks.
>
> - Ed
>
> # /usr/java/jdk1.6.0_25/bin/jmap -heap 11508
> Attaching to process ID 11508, please wait...
> Debugger attached successfully.
> Server compiler detected.
> JVM version is 20.0-b11
>
> using thread-local object allocation.
> Parallel GC with 13 thread(s)
>
> Heap Configuration:
>   MinHeapFreeRatio = 40
>   MaxHeapFreeRatio = 70
>   MaxHeapSize      = 6318718976 (6026.0MB)
>   NewSize          = 1310720 (1.25MB)
>   MaxNewSize       = 17592186044415 MB
>   OldSize          = 5439488 (5.1875MB)
>   NewRatio         = 2
>   SurvivorRatio    = 8
>   PermSize         = 21757952 (20.75MB)
>   MaxPermSize      = 85983232 (82.0MB)
>
> Heap Usage:
> PS Young Generation
> Eden Space:
>   capacity = 262144 (0.25MB)
>   used     = 132088 (0.12596893310546875MB)
>   free     = 130056 (0.12403106689453125MB)
>   50.3875732421875% used
> From Space:
>   capacity = 1900544 (1.8125MB)
>   used     = 163840 (0.15625MB)
>   free     = 1736704 (1.65625MB)
>   8.620689655172415% used
> To Space:
>   capacity = 1900544 (1.8125MB)
>   used     = 0 (0.0MB)
>   free     = 1900544 (1.8125MB)
>   0.0% used
> PS Old Generation
>   capacity = 4587520 (4.375MB)
>   used     = 2752448 (2.62493896484375MB)
>   free     = 1835072 (1.75006103515625MB)
>   59.998604910714285% used
> PS Perm Generation
>   capacity = 21757952 (20.75MB)
>   used     = 10180360 (9.708747863769531MB)
>   free     = 11577592 (11.041252136230469MB)
>   46.78914633141943% used
>
>
>
>
> On Thu, Nov 10, 2011 at 4:51 PM, Patrick Hunt <phunt@apache.org> wrote:
>
>> Perhaps easier to just use jmap?
>>
>> sudo jmap -heap <jvmpid>
>>
>> Patrick
>>
>> On Thu, Nov 10, 2011 at 3:49 PM, Ed Sexton <sextonhadoop@gmail.com> wrote:
>> > Hi Henry,
>> >
>> > I checked the process listing via ps and top, and saw VIRT=6477m  and
>> > RES=83m.
>> >
>> > How would one check if it is the jvm heap vs mapped space?
>> >
>> > Thanks as always Henry.
>> > Ed
>> >
>> > On Thu, Nov 10, 2011 at 2:04 PM, Henry Robinson <henry@cloudera.com>
>> wrote:
>> >
>> >> Ed - how are you making the observation that the heap size is 6477m? Is
>> it
>> >> from a profiler, or from top or similar? The amount of virtual memory
>> >> mapped for Java processes can often be very large, especially on recent
>> >> RHEL systems with the new arena allocator in glibc. Worth checking
>> whether
>> >> it's the JVM heap or the amount of mapped space.
>> >>
>> >> cheers,
>> >> Henry
>> >>
>> >> On 10 November 2011 13:47, Ed Sexton <sextonhadoop@gmail.com> wrote:
>> >>
>> >> > Hello Group-
>> >> >
>> >> > I am running CDH zookeeper version 3.3.3+12.12 and noticed that
>> zookeeper
>> >> > is running with a heapsize of 6477m.
>> >> >
>> >> > I start zookeeper standalone, via
>> /etc/init.d/hadoop-zookeeper-server.  I
>> >> > do not see reference to any ZOOKEEPER_HEAPIZE in the startup or
>> zoo.cfg,
>> >> > nor do I see any special JVMFLAGS in export/usr/bin/zookeeper-server,
>> >> which
>> >> > is what is called by the startup script, it just has this definition:
>> >> >
>> >> > JVMFLAGS=-Dzookeeper.log.threshold=INFO
>> >> >
>> >> > Can someone guide me to where the 6GB heapsize is being referenced?
>>  This
>> >> > seems a bit large of a heap.
>> >> >
>> >> > Thanks for your guidance.
>> >> >
>> >> > Sincerely,
>> >> > Ed
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Henry Robinson
>> >> Software Engineer
>> >> Cloudera
>> >> 415-994-6679
>> >>
>> >
>> >
>> >
>>
>

Mime
View raw message