mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammed Al khooja <mkho...@gmail.com>
Subject Mahout heap out of space
Date Tue, 15 Nov 2011 22:12:14 GMT
Hi,

I'm running Mahout LDA on a cluster but I'm getting the java out of memory
exception (below).  How do I increase the MAHOUT_HEAP variable in
bin\mahout.sh ?  When I add the line MAHOUT_HEAP=2000 it doesn't seem to
change !  Do I have to reload or rebuild anything so that changes take
effect ?


thanks.

2011-11-15 16:59:39,697 INFO org.apache.hadoop.util.NativeCodeLoader:
Loaded the native-hadoop library
2011-11-15 16:59:39,836 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2011-11-15 16:59:39,888 WARN org.apache.hadoop.conf.Configuration:
/scratch1/hadoop/mapred/local/taskTracker/mkhooja/jobcache/job_201110181830_3456/job.xml:a
attempt to override final parameter: dfs.data.dir; Ignoring.
2011-11-15 16:59:39,956 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb =
100
2011-11-15 16:59:40,001 INFO org.apache.hadoop.mapred.MapTask: data buffer
= 79691776/99614720
2011-11-15 16:59:40,001 INFO org.apache.hadoop.mapred.MapTask: record
buffer = 262144/327680
2011-11-15 16:59:48,863 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2011-11-15 16:59:48,866 FATAL org.apache.hadoop.mapred.Child: Error running
child : java.lang.OutOfMemoryError: Java heap space
at org.apache.mahout.math.DenseMatrix.<init>(DenseMatrix.java:51)
at
org.apache.mahout.clustering.lda.LDAInference.createPhiMatrix(LDAInference.java:154)
at org.apache.mahout.clustering.lda.LDAInference.infer(LDAInference.java:93)
at org.apache.mahout.clustering.lda.LDAMapper.map(LDAMapper.java:48)
at org.apache.mahout.clustering.lda.LDAMapper.map(LDAMapper.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)

-- 

M.khouja

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message