hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: JobTracker memory usage peaks once a day and OOM sometimes
Date Tue, 08 Feb 2011 20:56:27 GMT
As Allen mentioned it's really hard for anyone here to help you with  
CDH. You'll have to ask their user lists.

OTOH, there are significant enhancements we've done to the JobTracker  
in 0.21 and 0.22... the problem is that there isn't an Apache release  
yet.

the JT we run at Y! has a *huge* number of fixes for memory and  
performance (>100% improvement) which we've started putting into a  
0.20.100 branch - you might be interested in using that if you want a  
packaged release: http://people.apache.org/~acmurthy/hadoop-0.20.100-rc0/

Arun

On Feb 8, 2011, at 8:59 AM, Maxim Zizin wrote:

> Hi all,
>
> We monitor JT, NN and SNN memory usage and observe the following
> behavior in our Hadoop cluster. JT's heap size is set to 2000m.  
> About 18
> hours a day it uses ~1GB but every day roughly at the minute it was
> started its used memory increases to ~1.5GB and then decreases back to
> ~1GB in about 6 hours. Sometimes this takes a bit more than 6 hours,
> sometimes a bit less. I was wondering whether anyone here knows what  
> JT
> does once a day that makes it use 1.5 times more memory than normally.
>
> We're so interested in JT memory usage because during last two weeks  
> we
> twice had JT getting out of heap space. Both times right after those
> daily used memory peaks when it was going down from 1.5GB to 1GB it
> started increasing again until got stuck at ~2.2GB. After that it
> becomes unresponsive and we have to restart it.
>
> We're using Cloudera's CDH2 version 0.20.1+169.113.
>
> -- 
> Regards, Max


Mime
View raw message