hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: Java heap space error
Date Mon, 22 Oct 2012 18:01:25 GMT

Did this job ever run successfully for you? With 200m heap size?

Seems like your maps are failing. Can you paste your settings for the following:
 - io.sort.factor
 - io.sort.mb
 - mapreduce.map.sort.spill.percent

Thanks,
+Vinod

On Oct 21, 2012, at 6:18 AM, Subash D'Souza wrote:

> I'm running CDH 4 on  a 4 node cluster each with 96 G of RAM. Up until last week the
cluster was running until there was an error in the name node log file and I had to reformat
it put the data back 
> 
> Now when I run hive on YARN. I keep getting a Java heap space error. Based on the research
I did. I upped the my mapred.child.java.opts first from 200m to 400 m to 800m and I still
have the same issue. It seems to fail near the 100% mapper mark
> 
> I checked the log files and the only thing that it does output is java heap space error.
Nothing more.
> 
> Any help would be appreciated.
> 
> Thanks
> Subash
> 


Mime
View raw message