hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Pollak <...@athena.com>
Subject Re: OutOfMemory exception
Date Wed, 25 Oct 2006 18:50:12 GMT

As Emily Latella would say, "Never mind"

I kept upping the memory for mapred.child.java.opts until the problem  
went away.

Sorry for wasting bandwidth on my own stupidity.


On Oct 25, 2006, at 9:29 AM, David Pollak wrote:

> Howdy,
> I'm new to Hadoop.  I've got a network of 8 machines with ~1.8TB of  
> storage.  My first Hadoop test run is to count the URLs in a set of  
> crawled pages (~1.6M pages consuming about 70GB of space.)  When I  
> run my app (or just run the Grep example) on the data set, the map  
> task gets to 100%, then I get an IOException and when I review the  
> logs, there's an OutOfMemory error listed in the tasktracker logs  
> ("INFO org.apache.hadoop.madred.TaskRunner: task_0001_m_000258_0  
> java.lang.OutOfMemoryError: Java heap space.")
> I've tried upping mapred.child.java.opts, but that doesn't seem to  
> make a difference.
> Any suggestions on what I can do?
> Thanks,
> David

View raw message