hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Re: how to prevent JAVA HEAP OOM happen in shuffle process in a MR job?
Date Mon, 02 Dec 2013 19:08:12 GMT
version is rewally important here..

- If 1.x, then Where (NN , JT , TT ?)
- if 2.x, then where? (AM, NM, ... ?) <-- probably less likely here, since
the resources are ephemeral.

I know that some older 1x versions had an issue with the jobtracker having
an ever-expanding hashmap or something like that, so that if you ran 100s
of jobs, you could get OOM erros on the JobTracker.

View raw message