mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suneel Marthi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAHOUT-1129) Mahout Java Heap Out of Memory in TrainNewsGroup class
Date Tue, 18 Dec 2012 20:28:12 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535269#comment-13535269
] 

Suneel Marthi commented on MAHOUT-1129:
---------------------------------------

Did u try running the example by setting the JVM memory settings ?
                
> Mahout Java Heap Out of Memory in TrainNewsGroup class
> ------------------------------------------------------
>
>                 Key: MAHOUT-1129
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1129
>             Project: Mahout
>          Issue Type: Bug
>          Components: Classification, Examples
>    Affects Versions: 0.7
>         Environment: Linux - RHEL 5.6
>            Reporter: Huned Lokhandwala
>            Priority: Critical
>              Labels: test
>             Fix For: 0.7
>
>
> On Mahout on Linux RHEL 5.6 (Single node, no cluster), a Java Heap Space Out of Memory
Exception occurs when calling the TrainNewsGroup class from the mahout-examples-0.7.0.17-job.jar
file. I downloaded the 20news-bydate.tar.gz (from http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz
) to use as classification data and added it in the folder as below, and called the org.apache.mahout.classifier.sgd.TrainNewsGroups
on the folder and found the Java Heap Space Out of Memory Exception as show below.
> Output from run:
> > /usr/lib/mahout/bin/mahout org.apache.mahout.classifier.sgd.TrainNewsGroups /artifacts/mahout_sgd_classifier/20news-bydate-train
> Running on hadoop, using /usr/bin/hadoop and HADOOP_CONF_DIR=
> MAHOUT-JOB: /usr/lib/mahout/mahout-examples-0.7.0.17-job.jar
> 12/12/17 20:35:06 WARN driver.MahoutDriver: No org.apache.mahout.classifier.sgd.TrainNewsGroups.props
found on classpath, will use command-line arguments only
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.mahout.math.DenseMatrix.<init>(DenseMatrix.java:50)
> 	at org.apache.mahout.classifier.sgd.OnlineLogisticRegression.<init>(OnlineLogisticRegression.java:60)
> 	at org.apache.mahout.classifier.sgd.CrossFoldLearner.<init>(CrossFoldLearner.java:67)
> 	at org.apache.mahout.classifier.sgd.CrossFoldLearner.copy(CrossFoldLearner.java:217)
> 	at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$Wrapper.copy(AdaptiveLogisticRegression.java:399)
> 	at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression$Wrapper.copy(AdaptiveLogisticRegression.java:386)
> 	at org.apache.mahout.ep.State.copy(State.java:94)
> 	at org.apache.mahout.ep.State.mutate(State.java:114)
> 	at org.apache.mahout.ep.EvolutionaryProcess.initializePopulation(EvolutionaryProcess.java:103)
> 	at org.apache.mahout.ep.EvolutionaryProcess.<init>(EvolutionaryProcess.java:97)
> 	at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.setupOptimizer(AdaptiveLogisticRegression.java:279)
> 	at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.setPoolSize(AdaptiveLogisticRegression.java:265)
> 	at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.<init>(AdaptiveLogisticRegression.java:121)
> 	at org.apache.mahout.classifier.sgd.AdaptiveLogisticRegression.<init>(AdaptiveLogisticRegression.java:100)
> 	at org.apache.mahout.classifier.sgd.TrainNewsGroups.main(TrainNewsGroups.java:100)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> 	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> 	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message