hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bai Shen <baishen.li...@gmail.com>
Subject Re: Out of Memory during Reduce Merge
Date Tue, 10 Jan 2012 21:29:51 GMT
I set the following flags, as suggested, but I'm not seeing any output in
the specified directory.

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/data
-XX:+UseConcMarkSweepGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
-XX:MaxPermSize=512m -XX:+PrintTenuringDistribution

Why is the merge trying to load all of the data into memory?

On Tue, Jan 10, 2012 at 10:25 AM, Bai Shen <baishen.lists@gmail.com> wrote:

> I appear to be having the same problem as this poster.
> http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201006.mbox/%3C312639.9108.qm@web114409.mail.gq1.yahoo.com%3E
> Unfortunately, the thread seems to end with no resolution.  Does anyone
> know if this has been resolved?
> I'm using SCM 3.7.1 if that makes a difference.

View raw message