hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Pratt <prat...@adobe.com>
Subject RE: Hmaster and HRegionServer disappearance reason to ask
Date Mon, 16 Jul 2012 22:55:03 GMT
This sounds similar to something I've seen before, but in that case I found the winning GC
arguments to be something like

-XX:+UseParallelGC -XX:+UseParallelOldGC -XX:MaxDirectMemorySize=128m

(note the old gen parallel compacting collector rather than the ParNew collector which IIRC
is used with concurrent GC by default)

I don't recall the MaxDirectMemorySize on its own preventing massive off-heap memory allocations
from piling up.

Just my 2 cents, YMMV.


-----Original Message-----
From: Laxman [mailto:lakshman.ch@huawei.com] 
Sent: Wednesday, July 11, 2012 9:22 PM
To: 'Pablo Musa'; user@hbase.apache.org
Subject: RE: Hmaster and HRegionServer disappearance reason to ask

> > 1) Fix the direct memory usage to a fixed value -
> XX:MaxDirectMemorySize=1G
> This flag should be in RS ou DN?

We need to apply for both but limit can be increased based on your load (May be 2G).
Also we can to apply to all processes which are having following symptoms.

1) Allocated heap is few GB (4 to 8)
2) VIRT/RES will occupy double the heap (like 15GB) or even more
3) Long pauses in GC log (allocated heap is just <=8GB)
4) Your application uses lot of NIO/RMI calls(Ex: DataNode, RegionServer)

In our cluster we apply for all server processes (NN, DN, HM, RS, JT, TT, ZooKeeper).
Long pauses are disappeared after we set this flag (esp. for DN and RS).

View raw message