hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Azuryy Yu <azury...@gmail.com>
Subject Re: how to prevent JAVA HEAP OOM happen in shuffle process in a MR job?
Date Wed, 04 Dec 2013 01:38:14 GMT
Hi Jay,
Can you point out a detail about 1.x that job tracker having an
ever-expanding hash map. I want fix it.


On Tue, Dec 3, 2013 at 3:08 AM, Jay Vyas <jayunit100@gmail.com> wrote:

> version is rewally important here..
>
> - If 1.x, then Where (NN , JT , TT ?)
> - if 2.x, then where? (AM, NM, ... ?) <-- probably less likely here, since
> the resources are ephemeral.
>
> I know that some older 1x versions had an issue with the jobtracker having
> an ever-expanding hashmap or something like that, so that if you ran 100s
> of jobs, you could get OOM erros on the JobTracker.
>

Mime
View raw message