Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 234BE101C5 for ; Fri, 9 Jan 2015 16:49:15 +0000 (UTC) Received: (qmail 3875 invoked by uid 500); 9 Jan 2015 16:49:10 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 3778 invoked by uid 500); 9 Jan 2015 16:49:10 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 3739 invoked by uid 99); 9 Jan 2015 16:49:10 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Jan 2015 16:49:10 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mw@plista.com designates 77.220.236.183 as permitted sender) Received: from [77.220.236.183] (HELO mail.plista.com) (77.220.236.183) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Jan 2015 16:49:05 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.plista.com (Postfix) with ESMTP id 61F1F15957A for ; Fri, 9 Jan 2015 17:48:43 +0100 (CET) Received: from mail.plista.com ([127.0.0.1]) by localhost (mail.plista.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id LkSqY3pdxjse for ; Fri, 9 Jan 2015 17:48:43 +0100 (CET) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.plista.com (Postfix) with ESMTP id 3E16815961B for ; Fri, 9 Jan 2015 17:48:43 +0100 (CET) X-Virus-Scanned: amavisd-new at plista.com Received: from mail.plista.com ([127.0.0.1]) by localhost (mail.plista.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 9SeGWSE7cnxT for ; Fri, 9 Jan 2015 17:48:43 +0100 (CET) Received: from [192.168.2.160] (unknown [213.61.213.230]) by mail.plista.com (Postfix) with ESMTPSA id 0286315957A for ; Fri, 9 Jan 2015 17:48:42 +0100 (CET) Message-ID: <54B0066A.3040809@plista.com> Date: Fri, 09 Jan 2015 17:48:42 +0100 From: mw User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: user@hadoop.apache.org Subject: Re: Using Mahout 1.0-SNAPSHOT with yarn cluster References: <54AF9F99.3050103@plista.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------060107070603080505040003" X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------060107070603080505040003 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit I got it working. I took me a moment to figure out that tmpjars is actually an attribute :) Thank you very much! Best, Max On 01/09/2015 04:23 PM, Ted Yu wrote: > Pardon me, setJarByClass() should point to your class. > You can pass comma separated list of jars (including mahout jar) this way: > > conf.set("tmpjars", list-of-jars); > > Cheers > > > On Fri, Jan 9, 2015 at 7:17 AM, Ted Yu > wrote: > > In your driver, you can call: > > job.setJarByClass(theclass.class); > > where theclass can be one of the classes in mahout jar. > > FYI > > > On Fri, Jan 9, 2015 at 1:30 AM, mw > wrote: > > Hello, > > i am working on a web application that should execute lda on a > external yarn cluster. > > I am uploading all the relevant sequence files onto the yarn > cluter. > This is how it try to remotely execute lda on the cluster. > > try { > ugi.doAs(new PrivilegedExceptionAction() { > public Void run() throws Exception { > Configuration hdoopConf = new > Configuration(); > hdoopConf.set("fs.defaultFS", > "hdfs://xxx.xxx.xxx.xxx:9000/user/xx"); > hdoopConf.set("yarn.resourcemanager.hostname", > "xxx.xxx.xxx.xxx"); > hdoopConf.set("mapreduce.framework.name > ", "yarn"); > hdoopConf.set("mapred.framework.name > ", "yarn"); > hdoopConf.set("mapred.job.tracker", "xxx.xxx.xxx.xxx"); > hdoopConf.set("dfs.permissions.enabled", "false"); > hdoopConf.set("hadoop.job.ugi", "xx"); > hdoopConf.set("mapreduce.jobhistory.address","xxx.xxx.xxx.xxx:10020" > ); > CVB0Driver driver = new CVB0Driver(); > try { > driver.run(hdoopConf, > sparseVectorIn.suffix("/matrix"), > topicsOut, k, numTerms, > doc_topic_smoothening, term_topic_smoothening, > maxIter, iteration_block_size, > convergenceDelta, > sparseVectorIn.suffix("/dictionary.file-0"), > topicsOut.suffix("//DocumentTopics//"), sparseVectorIn, > seed, testFraction, > numTrainThreads, numUpdateThreads, maxItersPerDoc, > numReduceTasks, > backfillPerplexity); > } catch (ClassNotFoundException e) { > e.printStackTrace(); > } catch (InterruptedException e) { > e.printStackTrace(); > } > return null; > } > }); > } catch (InterruptedException e) { > e.printStackTrace(); > } > > I am getting the following error message: > > Error: java.lang.ClassNotFoundException: > org.apache.mahout.math.Vector > at java.net.URLClassLoader$1.run(URLClassLoader.java:372) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:360) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:344) > at > org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1844) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1809) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1903) > > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1929) > > at > org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:983) > > at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391) > at > org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:675) > at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > Error: java.lang.ClassNotFoundException: > org.apache.mahout.math.Vector > at java.net.URLClassLoader$1.run(URLClassLoader.java:372) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:360) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:344) > at > org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1844) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1809) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1903) > > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1929) > > at > org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:983) > > at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391) > at > org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:675) > at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > Error: java.lang.ClassNotFoundException: > org.apache.mahout.math.Vector > at java.net.URLClassLoader$1.run(URLClassLoader.java:372) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:360) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:344) > at > org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1844) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1809) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1903) > > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1929) > > at > org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:983) > > at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391) > at > org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:675) > at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > Error: java.lang.ClassNotFoundException: > org.apache.mahout.math.Vector > at java.net.URLClassLoader$1.run(URLClassLoader.java:372) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:360) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:344) > at > org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1844) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1809) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1903) > > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1929) > > at > org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:983) > > at > org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391) > at > org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80) > at > org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:675) > at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) > at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at > org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > java.lang.InterruptedException: Failed to complete iteration 1 > stage 1 > at > org.apache.mahout.clustering.lda.cvb.CVB0Driver.runIteration(CVB0Driver.java:502) > at > org.apache.mahout.clustering.lda.cvb.CVB0Driver.run(CVB0Driver.java:319) > > ... > > I was told on the user-mahout mailing list that the lda jar > mahout sents to yarn should contain all relevant classes. Any > idea why this error is thrown nevertheless? > > Best, > Max > > > --------------060107070603080505040003 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable I got it working. I took me a moment to figure out that tmpjars is actually an attribute :)
Thank you very much!

Best,
Max
On 01/09/2015 04:23 PM, Ted Yu wrote:<= br>
Pardon me,=C2=A0setJarByClass() should point to your class.
You can pass comma separated list of jars (including mahout jar) this way:

conf.set("tmpjars",=C2=A0= list-of-jars);

Cheers


On Fri, Jan 9, 2015 at 7:17 AM, Ted Yu <yuzhi= hong@gmail.com> wrote:
In your driver, you can call:

=C2=A0 =C2=A0=C2=A0job.setJarByClass(theclass.cl= ass);

where theclass can be one of the classes in mahout jar.

FYI


On Fri, Jan 9, 2015 at 1:30 AM, mw <m= w@plista.com> wrote:
Hello,
i am working on a web application that should execute lda on a external yarn cluster.

I am uploading all the relevant sequence files onto the yarn cluter.
This is how it try to remotely execute lda on the cluster.

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 try { =
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 ugi.doAs(new PrivilegedExceptionAction<Void>() {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 public Void run() throws Exception {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Configura= tion hdoopConf =3D new Configuration();
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf.set("fs.defaultFS", "hdfs://xxx.xxx.xxx.xxx:9000/user/xx");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf.set("yarn.resourcemanager.hostname", "xxx.xxx.xxx.xxx");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf= .set("mapreduce.framework.name"= , "yarn");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf= .set("mapred.framework.name", "yarn");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf.set("mapred.job.tracker", "xxx.xxx.xxx.xxx");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf.set("dfs.permissions.enabled", "false");
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hdoopConf.set("hadoop.job.ugi", "xx");
hdoopConf.set("mapreduce.jobhistory.address","xxx= .xxx.xxx.xxx:10020" );
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CVB0Drive= r driver =3D new CVB0Driver();
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 try { =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 driver.run(hdoopConf, sparseVectorIn.suffix("/matrix"),
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 topicsOut, k= , numTerms, doc_topic_smoothening, term_topic_smoothening,
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 maxIter, iteration_block_size, convergenceDelta,
sparseVectorIn.suffix("/dictionary.file-0"), topicsOut.suffix("/DocumentTopics= /"), sparseVectorIn,
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 seed, testFraction, numTrainThreads, numUpdateThreads, maxItersPerDoc,
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 numReduceTas= ks, backfillPerplexity);
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } catch (ClassNotFoundException e) {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 e.printStackTrace();
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } catch (InterruptedException e) {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 e.printStackTrace();
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return nu= ll;
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 });
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } catc= h (InterruptedException e) {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 e.printStackTrace();
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 }

I am getting the following error message:

Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :372)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :361)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader.findClass(URLClassLoader.= java:360)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 424)
=C2=A0=C2=A0=C2=A0=C2=A0at sun.misc.Launcher$AppClassLoader.loadClass(Launch= er.java:308)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 357)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e0(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e(Class.java:344)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.j= ava:1844)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:18= 09)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1903)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1929)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.= init(MapTask.java:983)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.access$100(MapTa= sk.java:80)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.= java:675)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.runNewMapper(Map= Task.java:747)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.run(MapTask.java= :340)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild$2.run(YarnChil= d.java:168)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at javax.security.auth.Subject.doAs(Subject.java:422= )
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation= .java:1614)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild.main(YarnChild= .java:163)

Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :372)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :361)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader.findClass(URLClassLoader.= java:360)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 424)
=C2=A0=C2=A0=C2=A0=C2=A0at sun.misc.Launcher$AppClassLoader.loadClass(Launch= er.java:308)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 357)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e0(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e(Class.java:344)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.j= ava:1844)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:18= 09)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1903)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1929)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.= init(MapTask.java:983)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.access$100(MapTa= sk.java:80)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.= java:675)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.runNewMapper(Map= Task.java:747)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.run(MapTask.java= :340)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild$2.run(YarnChil= d.java:168)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at javax.security.auth.Subject.doAs(Subject.java:422= )
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation= .java:1614)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild.main(YarnChild= .java:163)

Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :372)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :361)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader.findClass(URLClassLoader.= java:360)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 424)
=C2=A0=C2=A0=C2=A0=C2=A0at sun.misc.Launcher$AppClassLoader.loadClass(Launch= er.java:308)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 357)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e0(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e(Class.java:344)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.j= ava:1844)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:18= 09)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1903)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1929)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.= init(MapTask.java:983)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.access$100(MapTa= sk.java:80)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.= java:675)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.runNewMapper(Map= Task.java:747)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.run(MapTask.java= :340)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild$2.run(YarnChil= d.java:168)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at javax.security.auth.Subject.doAs(Subject.java:422= )
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation= .java:1614)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild.main(YarnChild= .java:163)

Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :372)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader$1.run(URLClassLoader.java= :361)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.net.URLClassLoader.findClass(URLClassLoader.= java:360)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 424)
=C2=A0=C2=A0=C2=A0=C2=A0at sun.misc.Launcher$AppClassLoader.loadClass(Launch= er.java:308)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.ClassLoader.loadClass(ClassLoader.java:= 357)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e0(Native Method)
=C2=A0=C2=A0=C2=A0=C2=A0at java.lang.Class.forNam= e(Class.java:344)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.j= ava:1844)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:18= 09)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1903)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.conf.Configuration.getClass(Con= figuration.java:1929)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:837)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.= init(MapTask.java:983)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391)=
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.access$100(MapTa= sk.java:80)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.= java:675)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.runNewMapper(Map= Task.java:747)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.MapTask.run(MapTask.java= :340)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild$2.run(YarnChil= d.java:168)
=C2=A0=C2=A0=C2=A0=C2=A0at java.security.AccessController.doPrivileged(Nativ= e Method)
=C2=A0=C2=A0=C2=A0=C2=A0at javax.security.auth.Subject.doAs(Subject.java:422= )
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation= .java:1614)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.hadoop.mapred.YarnChild.main(YarnChild= .java:163)

java.lang.InterruptedException: Failed to complete iteration 1 stage 1
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.mahout.clustering.lda.cvb.CVB0Driver.runIteration(CVB0Driver.j= ava:502)
=C2=A0=C2=A0=C2=A0=C2=A0at org.apache.mahout.clustering.lda.cvb.CVB0Driver.r= un(CVB0Driver.java:319)
=C2=A0=C2=A0=C2=A0 ...

I was told on the user-mahout mailing list that the lda jar mahout sents to yarn should contain all relevant classes. Any idea why this error is thrown nevertheless?

Best,
Max



--------------060107070603080505040003--