hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Zizin <mzi...@iponweb.net>
Subject Re: JobTracker memory usage peaks once a day and OOM sometimes
Date Tue, 08 Feb 2011 21:17:33 GMT
Arun,

Thanks for an advice. I will definitely consider using that release.

On 2/8/2011 11:56 PM, Arun C Murthy wrote:
> As Allen mentioned it's really hard for anyone here to help you with 
> CDH. You'll have to ask their user lists.
>
> OTOH, there are significant enhancements we've done to the JobTracker 
> in 0.21 and 0.22... the problem is that there isn't an Apache release 
> yet.
>
> the JT we run at Y! has a *huge* number of fixes for memory and 
> performance (>100% improvement) which we've started putting into a 
> 0.20.100 branch - you might be interested in using that if you want a 
> packaged release: http://people.apache.org/~acmurthy/hadoop-0.20.100-rc0/
>
> Arun
>
> On Feb 8, 2011, at 8:59 AM, Maxim Zizin wrote:
>
>> Hi all,
>>
>> We monitor JT, NN and SNN memory usage and observe the following
>> behavior in our Hadoop cluster. JT's heap size is set to 2000m. About 18
>> hours a day it uses ~1GB but every day roughly at the minute it was
>> started its used memory increases to ~1.5GB and then decreases back to
>> ~1GB in about 6 hours. Sometimes this takes a bit more than 6 hours,
>> sometimes a bit less. I was wondering whether anyone here knows what JT
>> does once a day that makes it use 1.5 times more memory than normally.
>>
>> We're so interested in JT memory usage because during last two weeks we
>> twice had JT getting out of heap space. Both times right after those
>> daily used memory peaks when it was going down from 1.5GB to 1GB it
>> started increasing again until got stuck at ~2.2GB. After that it
>> becomes unresponsive and we have to restart it.
>>
>> We're using Cloudera's CDH2 version 0.20.1+169.113.
>>
>> -- 
>> Regards, Max
>

-- 
Regards, Max


Mime
View raw message