hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: Tuning question...
Date Wed, 07 Apr 2010 20:41:09 GMT

The JVM will auto-tune many of these params.  For example it will set
the parallel GC threads = # of cpus.  Unless you are having a specific
problem I wouldn't touch that value.

As for the newSize... the argument is like such.  HBase is constantly
tenuring tons of data (Block cache is the primary culprit here) and
left to it's own devices, the JVM would increase the new size until
the "minor" gc pauses would take something like 800ms or maybe more
(!!).  So by restricting the new size we are tenuring more data
frequently, losing the benefit of generational GC, but keeping overall
GC pause low.  It's a trade off which is forced on us by the way the
block cache works right now.  A different design of the block cache
could conceivably help here.

I use this in prod:
"-XX:+DoEscapeAnalysis -XX:+AggressiveOpts -XX:+UseConcMarkSweepGC
-XX:NewSize=64m -XX:MaxNewSize=64m
-XX:CMSInitiatingOccupancyFraction=88 -verbose:gc -XX:+PrintGCDetails

and things work reasonably well.

On Wed, Apr 7, 2010 at 1:35 PM, Michael Segel <michael_segel@hotmail.com> wrote:
> Hi,
> I was trying to tune my HBase environment, following the stuff on the wiki and I have
a question regarding the GC parallel threads setting.
> Is there a good 'rule of thumb' or formula like 1 thread per core, or something?
> Sorry I'm going from memory...-XX:ParallelGCThreads=<number_of_GC_threads>.
> Also what about the newSize, maxNewSize values? The example says 6M but is there any
other things to consider?
> Thx
> -Mike
> _________________________________________________________________
> The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail.
> http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4

View raw message