mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject RE: java heap space exception using org.apache.mahout.fpm.pfpgrowth.FPGrowthDriver
Date Wed, 09 Feb 2011 01:40:52 GMT
Looks like the heap size for your mapred jvms is too small. What version of hadoop are you
using? You need to set (for hadoop 0.20.2) variable to something reasonable.
14K documents is very small dataset but not sure what your child jvm heap size is set to.

Set it to something like 512m or higher and try again.

From: ext Yang Sun []
Sent: Monday, February 07, 2011 2:55 PM
Subject: java heap space exception using org.apache.mahout.fpm.pfpgrowth.FPGrowthDriver


I'm trying to use parallel FPGrowth on a text based data set with
about 14K documents. But when I run mahout, I got the following

FATAL org.apache.hadoop.mapred.TaskTracker: Error running child :
java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(
        at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(
        at org.apache.hadoop.mapred.MapTask.runNewMapper(
        at org.apache.hadoop.mapred.Child.main(

The command:

hadoop jar mahout-examples-0.5-SNAPSHOT-job.jar
org.apache.mahout.fpm.pfpgrowth.FPGrowthDriver -i tital_tokens -o
patterns -k 3000 -method mapreduce -g 10 -regex '[\ ]' -s 10

Can someone tell me how I can fix this? Or is it possible to use the
algorithm for a text based dataset?



View raw message