hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Region Servers Crashing during Random Reads
Date Fri, 04 Feb 2011 16:21:18 GMT
On Fri, Feb 4, 2011 at 12:20 AM, Lars George <lars.george@gmail.com> wrote:
> I saw the -XX:MaxGCPauseMillis option too and assumed it is not that
> effective as it was never suggested so far. So it was simply not tried
> yet and someone has to be the guinea pig?
>

Yeah, haven't had good experience with these upper-bounds in pause
time.  We can add it though.


> So that 6M NewSize in the Wiki was OK previously and now 256M is
> actually better? The represent opposite ideas and I still try to
> understand which is better now, small eden or reasonably large one?
>

This is best thing I've read on GC:
http://www.scribd.com/doc/37127094/GCTuningPresentationFISL10

As I read it, avoid promotion to old gen if at all possible.


> Setting the -Xmn fixed seems to make sense given that it does do OK
> with various heap size and a ration may be too liberal giving away
> space.
>

256M seems way small still given what I read above.  Woud also like to
avoid hardcoding so its 256M whether user runs with 1G -- the default
or 8G.  If we set NewRatio to 32, thats about 256M new gen in 8G heap
(http://blogs.sun.com/jonthecollector/entry/the_second_most_important_gc).
 How about we set it to 16.  Then young gen is about 5%.

How about:

-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC
-XX:NewRatio=16
-XX:CMSInitiatingOccupancyFraction=70
-XX:+UseCMSInitiatingOccupancyOnly
-XX:MaxGCPauseMillis=100

We can add above to your auto-config patch Todd.  Yes, Lars, would
like to pull into the book but would also add note into hbase-env.sh
to say this is just starting point .... and we'd let go of the wiki
(it needs updating to reference Todd's new LZO fixings too).  If above
looks reasonable, I'll test it.

St.Ack

St.Ack

Mime
View raw message