mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kalmohsen <cstudent...@gmail.com>
Subject Errors while using Mahout 0.9 with Hadoop 2.4.0
Date Wed, 03 Sep 2014 07:57:53 GMT
Hello all

I am a master student who is willing to implement a scalable recommender 
system using Mahout, hadoop and spark as my dissertation. Furthermore, I 
configured Hadoop in a pseudo-distributed mode on my Macbook Pro and 
install eclipse with m2eclipse plugin to assist me in implementing mahout 
based projects. 

I’ve added Mahout0.9  and Hadoop 2.4.0 dependencies to my maven project in 
eclipse and things were OK when I tried to execute it on my machine as a 
single node (in non-distributed mode). However, as I tried to execute it 
on hadoop server using: 

Bin/hadoop jar itemBasedRecommender.jar  
org.apache.mahut.cf.taste.hadoop.itemRecommenderJob –input input/links-
converted.txt –output output –n 15 –s SIMILARITY_LOGLIKELIHOOD –
booleanData TRUE 

I got an error message saying:

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found 
interface org.apache.hadoop.mapreduce.JobContext, but class was expected

at org.apache.mahout.common.HadoopUtil.getCustomJobName
(HadoopUtil.java:174)

at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614)

at 
org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.ru
n(PreparePreferenceMatrixJob.java:73)
 
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run
(RecommenderJob.java:164)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main
(RecommenderJob.java:322)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
 
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
 
at java.lang.reflect.Method.invoke(Method.java:483)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

 

I search the web for a solution and I got to know that there is a 
compatibility problem with mahout 0.9 and hadoop 2.4.0. Some were saying 
that I have to download patches 1329, apply them and then re-compile 
mahout. I tried that but patches failed as well! 

Khadijas-MacBook-Pro:mahout-mahout-0.9 khadijaalmohsen$ patch -i 1329-
3.patch

patching file pom.xml

Hunk #1 FAILED at 213.

Hunk #2 FAILED at 227.

2 out of 2 hunks FAILED -- saving rejects to file pom.xml.rej

patching file pom.xml

Hunk #1 FAILED at 35.

Hunk #2 FAILED at 144.

Hunk #3 FAILED at 195.

3 out of 3 hunks FAILED -- saving rejects to file pom.xml.rej

patching file pom.xml

Khadijas-MacBook-Pro:mahout-mahout-0.9 khadijaalmohsen$ patch -i 1329-3-
additional.patch

patching file pom.xml

Hunk #1 succeeded at 676 with fuzz 1 (offset 460 lines).

Hunk #2 succeeded at 751 with fuzz 1 (offset 521 lines).

patching file pom.xml

Hunk #2 FAILED at 677.

Hunk #3 FAILED at 752.

2 out of 6 hunks FAILED -- saving rejects to file pom.xml.rej

patching file pom.xml

Hunk #1 FAILED at 201.

1 out of 1 hunk FAILED -- saving rejects to file pom.xml.rej

 

Some resources were advising to simply re-compile mahout. Thus, I tried 
that using several variation of mvn command:

mvn clean install –U

mvn clean install –Dhadoop2 –Dhadoop2.version=2.4.0

 but again I got an error :

…………………

testPrime(org.apache.mahout.math.random.MultinomialTest) Time elapsed: 
0.041 sec <<< FAILURE!

java.lang.AssertionError: expected:<16> but was:<17>

at __randomizedtesting.SeedInfo.seed([113DA6AF561FD9C7:2689253B3FD58807]:0)

at org.junit.Assert.fail(Assert.java:88)

at org.junit.Assert.failNotEquals(Assert.java:743)
 
at org.junit.Assert.assertEquals(Assert.java:118)

at org.junit.Assert.assertEquals(Assert.java:555)

at org.junit.Assert.assertEquals(Assert.java:542)

at org.apache.mahout.math.random.MultinomialTest.testPrime
(MultinomialTest.java:143)

………………………

[INFO] --------------------------------------------------------------------
----
 
[INFO] Reactor Summary:

[INFO] 

[INFO] Mahout Build Tools ................................ SUCCESS [ 1.898 
s]

[INFO] Apache Mahout ..................................... SUCCESS [ 0.516 
s]

[INFO] Mahout Math ....................................... FAILURE [01:41 
min]

[INFO] Mahout Core ....................................... SKIPPED

[INFO] Mahout Integration ................................ SKIPPED

[INFO] Mahout Examples ................................... SKIPPED

[INFO] Mahout Release Package ............................ SKIPPED

[INFO] Mahout Math/Scala wrappers ........................ SKIPPED

[INFO] --------------------------------------------------------------------
----

[INFO] BUILD FAILURE

[INFO] --------------------------------------------------------------------
----

[INFO] Total time: 01:44 min

[INFO] Finished at: 2014-09-03T08:25:35+03:00

[INFO] Final Memory: 28M/191M

[INFO] --------------------------------------------------------------------
----

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-
plugin:2.16:test (default-test) on project mahout-math: There are test 
failures.

[ERROR]

 

And now I do not know what to do and how to proceed further. Your 
cooperation is highly appreciated since I have only few months to finish 
my dissertation and I am struggling.

Thank you in advance

Regards,

kalmohsen



Mime
View raw message