hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Srinivas <sur...@hortonworks.com>
Subject Re: NN Memory Jumps every 1 1/2 hours
Date Sun, 23 Dec 2012 03:23:44 GMT
This looks to me is because of larger default young generation size in
newer java releases - see
I can see looking at your GC logs, around 6G space being used for young
generation (though I do not see logs related to minor collection). That
means for the same given number of objects, you have smaller old generation
space and hence old generation collection can no longer perform well.

It is unfortunate that such changes are made in java and that causes
previously working applications to fail. My suggestion is to not depend on
default young generation sizes any more. At large JVM sizes, the defaults
chosen by the JDK no longer works well. So I suggest protecting yourself
from such changes by explicitly specifying young generation size. Given my
experience of tuning GC at Yahoo clusters, at the number of objects you
have and total heap size you are allocating, I suggest setting the young
generation to 1G.

You can do that by adding

-XX:NewSize=1G -XX:MaxNewSize=1G

Let me know how it goes.

On Sat, Dec 22, 2012 at 5:59 PM, Edward Capriolo <edlinuxguru@gmail.com>wrote:

> 6333.934: [Full GC 10391746K->9722532K(17194656K), 63.9812940 secs]


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message