mahout-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Mahout > Why I can't pass the test of ALS-WR with the moivelens dataset
Date Mon, 03 Sep 2012 06:51:17 GMT
Space: Apache Mahout (https://cwiki.apache.org/confluence/display/MAHOUT)
Page: Why I can't pass the test of ALS-WR with the moivelens dataset (https://cwiki.apache.org/confluence/display/MAHOUT/2012/09/03/Why+I+can%27t+pass+the+test+of+ALS-WR+with+the+moivelens+dataset)

Added by Yuanshuai Sun:
---------------------------------------------------------------------
I deploy the mahout code in my machine according to the quickstart. But it throws NullPointerException,
just as follows

dm@cloud:~/workspace/mahout/examples/bin$ sh factorize-movielens-1M.sh path/to/ratings.dat

creating work directory at /tmp/mahout-work-dm
Converting ratings...
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.

Exception in thread "main" java.lang.NullPointerException
	at org.apache.mahout.common.AbstractJob.getOption(AbstractJob.java:385)
	at org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter.run(DatasetSplitter.java:75)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
	at org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter.main(DatasetSplitter.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.

12/09/03 14:45:21 INFO common.AbstractJob: Command line arguments: {--alpha=[40], --endPhase=[2147483647],
--implicitFeedback=[false], --input=[/tmp/mahout-work-dm/dataset/trainingSet/], --lambda=[0.065],
--numFeatures=[20], --numIterations=[10], --output=[/tmp/mahout-work-dm/als/out], --startPhase=[0],
--tempDir=[/tmp/mahout-work-dm/als/tmp]}
****/tmp/mahout-work-dm/dataset/trainingSet
12/09/03 14:45:22 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/dm/.staging/job_201209030824_0013
12/09/03 14:45:22 ERROR security.UserGroupInformation: PriviledgedActionException as:dm cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException:
Input path does not exist: /tmp/mahout-work-dm/dataset/trainingSet
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input
path does not exist: /tmp/mahout-work-dm/dataset/trainingSet
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
	at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
	at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
	at org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob.run(ParallelALSFactorizationJob.java:137)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
	at org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob.main(ParallelALSFactorizationJob.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.

12/09/03 14:45:24 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647],
--input=[/tmp/mahout-work-dm/dataset/probeSet/], --itemFeatures=[/tmp/mahout-work-dm/als/out/M/],
--output=[/tmp/mahout-work-dm/als/rmse/], --startPhase=[0], --tempDir=[/tmp/mahout-work-dm/als/tmp],
--userFeatures=[/tmp/mahout-work-dm/als/out/U/]}
****/tmp/mahout-work-dm/dataset/probeSet
12/09/03 14:45:25 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/dm/.staging/job_201209030824_0014
12/09/03 14:45:25 ERROR security.UserGroupInformation: PriviledgedActionException as:dm cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException:
Input path does not exist: /tmp/mahout-work-dm/dataset/probeSet
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input
path does not exist: /tmp/mahout-work-dm/dataset/probeSet
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
	at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
	at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
	at org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator.run(FactorizationEvaluator.java:92)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
	at org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator.main(FactorizationEvaluator.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Warning: $HADOOP_HOME is deprecated.

Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.

12/09/03 14:45:27 INFO common.AbstractJob: Command line arguments: {--endPhase=[2147483647],
--input=[/tmp/mahout-work-dm/als/out/userRatings/], --itemFeatures=[/tmp/mahout-work-dm/als/out/M/],
--maxRating=[5], --numRecommendations=[6], --output=[/tmp/mahout-work-dm/recommendations/],
--startPhase=[0], --tempDir=[temp], --userFeatures=[/tmp/mahout-work-dm/als/out/U/]}
****/tmp/mahout-work-dm/als/out/userRatings
12/09/03 14:45:28 INFO mapred.JobClient: Cleaning up the staging area hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/dm/.staging/job_201209030824_0015
12/09/03 14:45:28 ERROR security.UserGroupInformation: PriviledgedActionException as:dm cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException:
Input path does not exist: /tmp/mahout-work-dm/als/out/userRatings
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input
path does not exist: /tmp/mahout-work-dm/als/out/userRatings
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
	at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
	at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
	at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
	at org.apache.mahout.cf.taste.hadoop.als.RecommenderJob.run(RecommenderJob.java:95)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
	at org.apache.mahout.cf.taste.hadoop.als.RecommenderJob.main(RecommenderJob.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
-e 
RMSE is:

cat: /tmp/mahout-work-dm/als/rmse/rmse.txt: 没有那个文件或目录
-e 

-e 
Sample recommendations:

shuf: /tmp/mahout-work-dm/recommendations/part-m-00000: 没有那个文件或目录
-e 




Change your notification preferences: https://cwiki.apache.org/confluence/users/viewnotifications.action

Mime
View raw message