mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Wignall <Martin.Wign...@autotrader.co.uk>
Subject RE: error running a RecommenderJob
Date Wed, 18 Jan 2012 17:07:47 GMT
Sean,
When I execute my recommenderJob, with

hadoop jar target/mahout-core-0.6-SNAPSHOT-job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob
-Dmapred.input.dir=input/input.txt -Dmapred.output.dir=output --usersFile input/users.txt
--booleanData --similarityClassname SIMILARITY_COOCCURRENCE

…then I see that 7 jobs are successful, but 3 jobs fail (http://localhost:50030/jobtracker.jsp)
with the following output:

JOB NAME: PreparePreferenceMatrixJob-ItemIDIndexMapper-Reducer
DIAGNOSTIC INFO:  # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask:
task_201201171740_0001_m_000000

JOB NAME: PreparePreferenceMatrixJob-ToItemPrefsMapper-Reducer
DIAGNOSTIC INFO: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask:
task_201201171740_0002_m_000000

JOB NAME: RowSimilarityJob-CooccurrencesMapper-Reducer
DIAGNOSTIC INFO:  # of failed Reduce Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask:
task_201201171740_0005_r_000000


Any idea what I can do about this?

Many thanks once again!

Martin




-----Original Message-----
From: Martin Wignall
Sent: 18 January 2012 16:39
To: 'user@mahout.apache.org'
Subject: RE: error running a RecommenderJob

Hi Sean,
I've downloaded Mahout from scratch and built again. This time all is fine.

Originally I had changed a pom.xml file in order to get Mahout to build, but for some reason
this was not necessary on my second attempt. So all fine, thanks very much for your help.

Cheers!

Martin


-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: 18 January 2012 14:15
To: user@mahout.apache.org
Subject: Re: error running a RecommenderJob

The Hadoop version isn't the issue as it's a ClassNotFoundException for a Mahout dependency
for sure.

Hmm I do think maybe the pom change is relevant. The original error concerns some plugin,
org.eclipse.m2e, but that is not present in the project poms. are you sure you have the latest
files from SVN, no possiblity of other files, no other modifications? This plugin does indeed
not exist, but I don't see any use of it when I build. That seems to be the source of the
issue.

On Wed, Jan 18, 2012 at 1:55 PM, Martin Wignall < Martin.Wignall@autotrader.co.uk<mailto:Martin.Wignall@autotrader.co.uk>>
wrote:

> Thanks for your response Sean. Only other thing I can think of is I
> had to amend my pom.xml - I removed
>
> <plugin>
>  <groupId>org.eclipse.m2e</groupId>
>  <artifactId>lifecycle-mapping</artifactId>
>  <version>1.0.0</version>
>  <configuration>
>  <lifecycleMappingMetadata>
>  <pluginExecutions>
>  <pluginExecution>
>  <pluginExecutionFilter>
>  <groupId>org.apache.maven.plugins</groupId>
>  <artifactId>maven-dependency-plugin</artifactId>
>  <versionRange>[2.0,)</versionRange>
>  <goals>
>  <goal>copy-dependencies</goal>
>  </goals>
>  </pluginExecutionFilter>
>  <action>
>  <ignore />
>  </action>
>  </pluginExecution>
>  </pluginExecutions>
>  </lifecycleMappingMetadata>
>  </configuration>
>  </plugin>
>
> from <plugins>, because
>
> mvn clean package
>
> did not work from the command line. I got the following error:
>
> mvn clean package
> [INFO] Scanning for projects...
> [INFO]
> ----------------------------------------------------------------------
> --
> [INFO] Building Mahout Core
> [INFO]    task-segment: [clean, package]
> [INFO]
> ----------------------------------------------------------------------
> --
> Downloading:
> http://repo1.maven.org/maven2/org/eclipse/m2e/lifecycle-mapping/1.0.0/
> lifecycle-mapping-1.0.0.pom
> [INFO] Unable to find resource
> 'org.eclipse.m2e:lifecycle-mapping:pom:1.0.0' in repository central (
> http://repo1.maven.org/maven2)
> Downloading:
> http://repo1.maven.org/maven2/org/eclipse/m2e/lifecycle-mapping/1.0.0/
> lifecycle-mapping-1.0.0.pom
> [INFO] Unable to find resource
> 'org.eclipse.m2e:lifecycle-mapping:pom:1.0.0' in repository central (
> http://repo1.maven.org/maven2)
> [INFO]
> ----------------------------------------------------------------------
> --
> [ERROR] BUILD ERROR
> [INFO]
> ----------------------------------------------------------------------
> -- [INFO] Error building POM (may not be this project's POM).
>
>
> Project ID: org.eclipse.m2e:lifecycle-mapping
>
> Reason: POM 'org.eclipse.m2e:lifecycle-mapping' not found in repository:
> Unable to download the artifact from any repository
>
>  org.eclipse.m2e:lifecycle-mapping:pom:1.0.0
>
> from the specified remote repositories:
>  apache.snapshots (http://repository.apache.org/snapshots),
>  central (http://repo1.maven.org/maven2)
>
>  for project org.eclipse.m2e:lifecycle-mapping
>
>
> The above change to my pom.xml fixed the problem and allowed me to
> package. I don’t imagine that this is related. Do you? (Just trying to
> rule it out)
>
>
>
> In answer to your questions, I’m on maven version 2.2.1, and Ubuntu
> 11.10
>
> mvn --version
> Apache Maven 2.2.1 (rdebian-6)
> Java version: 1.6.0_23
> Java home: /usr/lib/jvm/java-6-openjdk/jre Default locale: en_GB,
> platform encoding: UTF-8 OS name: "linux" version:
> "3.0.0-14-generic-pae" arch: "i386" Family:
> "unix"
>
>
> And in terms of my mahout structure, I just downloaded trunk:
>
> martin@ubuntu:~/mahout/trunk$ ll
> total 168
> drwxrwxr-x 14 martin martin  4096 2012-01-09 16:53 ./ drwxrwxr-x  3
> martin martin  4096 2012-01-06 16:10 ../ drwxrwxr-x  3 martin martin
> 4096 2012-01-06 16:11 bin/ drwxrwxr-x  6 martin martin  4096
> 2012-01-09 15:54 buildtools/
> -rw-rw-r--  1 martin martin 14653 2012-01-09 17:48 .classpath
> drwxrwxr-x  8 martin martin  4096 2012-01-18 12:24 core/ drwxrwxr-x  5
> martin martin  4096 2012-01-09 15:54 distribution/
> -rw-rw-r--  1 martin martin  2320 2012-01-06 16:11 doap_Mahout.rdf
> drwxrwxr-x  7 martin martin  4096 2012-01-09 15:54 examples/
> -rw-rw-r--  1 martin martin   345 2012-01-06 16:11 .gitignore
> drwxrwxr-x  8 martin martin  4096 2012-01-09 15:54 integration/
> -rw-rw-r--  1 martin martin 39588 2012-01-06 16:10 LICENSE.txt
> drwxrwxr-x  7 martin martin  4096 2012-01-09 16:01 math/ drwxrwxr-x  5
> martin martin  4096 2012-01-10 11:22 mw-play/
> -rw-rw-r--  1 martin martin  1888 2012-01-06 16:10 NOTICE.txt
> -rw-rw-r--  1 martin martin 32289 2012-01-06 16:11 pom.xml
> -rw-rw-r--  1 martin martin   535 2012-01-09 16:41 .project
> -rw-rw-r--  1 martin martin  1655 2012-01-09 16:00 .project~
> -rw-rw-r--  1 martin martin  1200 2012-01-06 16:11 README.txt
> drwxrwxr-x  2 martin martin  4096 2012-01-09 15:58 .settings/
> drwxrwxr-x  5 martin martin  4096 2012-01-06 16:10 src/ drwxrwxr-x  6
> martin martin  4096 2012-01-06 16:10 .svn/ drwxrwxr-x  3 martin martin
> 4096 2012-01-06 16:10 target/
>
>
>
> Only other thing I can think of is I have hadoop version
> hadoop-0.20.205.0. (I say that because the error I see is when running
> hadoop after all…
>
> hadoop jar target/mahout-core-0.6-SNAPSHOT.jar
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob
> -Dmapred.input.dir=input/input.txt
> -Dmapred.output.dir=output --usersFile input/users.txt --booleanData
>
> gives
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/cli2/Option
>       at java.lang.Class.forName0(Native Method)
>       at java.lang.Class.forName(Class.java:264)
>       at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.cli2.Option
>       at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>       at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>       at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>       ... 3 more
> )
>
>
> Anything in my above setup look incorrect?
>
> Thanks,
>
> Martin Wignall
> Application Architect
> Trader Media Digital
> Redwood House
> Woodlands Business Park
> Newton-le-Willows
> WA12 0HE
> 01925 296 245
> 07525 405 281
> martin.wignall@autotrader.co.uk<mailto:martin.wignall@autotrader.co.uk>
>
>
> -----Original Message-----
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: 18 January 2012 13:04
> To: user@mahout.apache.org
> Subject: Re: error running a RecommenderJob
>
> The exact same comment generates the job file for me... I see a lot
> more output though, as it does the work of packaging the job file.
>
> I wonder if it's a Maven version thing (I use 3.0.3 on OS X 10.7) or,
> do you have all the other files in the project, above core/? I'd
> expect it to just outright fail if either of those were at issue though.
>
> Run package from the project root? wild guess. That shouldn't be needed.
>
> On Wed, Jan 18, 2012 at 12:51 PM, Martin Wignall <
> Martin.Wignall@autotrader.co.uk<mailto:Martin.Wignall@autotrader.co.uk<mailto:Martin.Wignall@autotrader.co.uk<mailto:Martin.Wignall@autotrader.co.uk>
> >>
> wrote:
>
> > Thanks Sean. Any idea why:
> >
> > mvn -DskipTests clean package
> >
> > (or
> > mvn -DskipTests package
> > )
> >
> > might not have created the .job file? Is there something else I need
> > to
> do?
> >
> >
>
>
> ________________________________________
>
>
> This e-mail is sent on behalf of Trader Media Group Limited,
> Registered
> Office: Auto Trader House, Cutbush Park Industrial Estate, Danehill,
> Lower Earley, Reading, Berkshire, RG6 4UT(Registered in England No. 4768833).
> This email and any files transmitted with it are confidential and may
> be legally privileged, and intended solely for the use of the
> individual or entity to whom they are addressed. If you have received
> this email in error please notify the sender. This email message has
> been swept for the presence of computer viruses.
>
>


________________________________________


This e-mail is sent on behalf of Trader Media Group Limited, Registered Office: Auto Trader
House, Cutbush Park Industrial Estate, Danehill, Lower Earley, Reading, Berkshire, RG6 4UT(Registered
in England No. 4768833). This email and any files transmitted with it are confidential and
may be legally privileged, and intended solely for the use of the individual or entity to
whom they are addressed. If you have received this email in error please notify the sender.
This email message has been swept for the presence of computer viruses. 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message