Return-Path: X-Original-To: apmail-mahout-user-archive@www.apache.org Delivered-To: apmail-mahout-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 398549178 for ; Wed, 15 Feb 2012 03:40:39 +0000 (UTC) Received: (qmail 33478 invoked by uid 500); 15 Feb 2012 03:40:37 -0000 Delivered-To: apmail-mahout-user-archive@mahout.apache.org Received: (qmail 33414 invoked by uid 500); 15 Feb 2012 03:40:37 -0000 Mailing-List: contact user-help@mahout.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@mahout.apache.org Delivered-To: mailing list user@mahout.apache.org Received: (qmail 32904 invoked by uid 99); 15 Feb 2012 03:40:28 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Feb 2012 03:40:28 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of goksron@gmail.com designates 209.85.210.170 as permitted sender) Received: from [209.85.210.170] (HELO mail-iy0-f170.google.com) (209.85.210.170) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Feb 2012 03:40:20 +0000 Received: by iakk32 with SMTP id k32so1488131iak.1 for ; Tue, 14 Feb 2012 19:39:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=oTL83iuwQ3GZdxXO1Q+qjikauVoVrqCCCXFeg7C/Grc=; b=S96BpWxIF/gOwB7SBnOtmwkeNE8PxCai24DuMcTEMYk3A1ARcaBnHoF2R9MH48duCM rs+SMml6kTmFteqLF5VX5GAP4huafTpQeMQesWzf9KHt6GiUGg6zs63iiMB1HY0RlRbu bHVp57CK2nO4cuQTJpL3HhgqXGHvpSLShzRWY= MIME-Version: 1.0 Received: by 10.42.157.65 with SMTP id c1mr31132074icx.42.1329277199389; Tue, 14 Feb 2012 19:39:59 -0800 (PST) Received: by 10.50.55.4 with HTTP; Tue, 14 Feb 2012 19:39:59 -0800 (PST) In-Reply-To: References: <5CB91E85-631C-4D4D-A21E-10D7A493CC9A@yahoo.com> Date: Tue, 14 Feb 2012 19:39:59 -0800 Message-ID: Subject: Re: Mahout 0.5 java.lang.IllegalStateException: No clusters found. Check your -c path. From: Lance Norskog To: user@mahout.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org See the other mail thread for the MAHOUT-504 JIRA. That jira is closed and fixed. The problem is that the program needs one of a few different combinations of arguments. It does not give you an error message describing the problem. On Tue, Feb 14, 2012 at 6:59 PM, Qiang Xu wrote: > > The new test is using command =C2=A0./bin/mahout kmeans -i =C2=A0examples= /bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c =C2=A0examples/bin/wo= rk/clusters -o =C2=A0examples/bin/work/reuters-kmeans -x 10 =C2=A0-ow -cl > Still the same problem. > >> From: xxqonline@hotmail.com >> To: user@mahout.apache.org >> Subject: RE: Mahout 0.5 java.lang.IllegalStateException: No clusters fou= nd. Check your -c path. >> Date: Wed, 15 Feb 2012 10:58:25 +0800 >> >> >> I have checked the command line: >> --clustering (-cl) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 If present, run clustering after >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0the iterations have taken place >> And try it, it seems the same behavior, could you give me more clue? >> op_cluster/hadoop-0.20.2/ >> HADOOP_CONF_DIR=3D/data/hadoop_cluster/hadoop-0.20.2/conf/ >> 12/02/15 11:16:23 INFO common.AbstractJob: Command line arguments: {--cl= ustering=3Dnull, --clusters=3Dexamples/bin/work/clusters, --convergenceDelt= a=3D0.5, --distanceMeasure=3Dorg.apache.mahout.common.distance.SquaredEucli= deanDistanceMeasure, --endPhase=3D2147483647, --input=3Dexamples/bin/work/r= euters-out-seqdir-sparse/tfidf-vectors/, --maxIter=3D10, --method=3Dmapredu= ce, --output=3Dexamples/bin/work/reuters-kmeans, --overwrite=3Dnull, --star= tPhase=3D0, --tempDir=3Dtemp} >> 12/02/15 11:16:23 INFO common.HadoopUtil: Deleting examples/bin/work/reu= ters-kmeans >> 12/02/15 11:16:23 INFO kmeans.KMeansDriver: Input: examples/bin/work/reu= ters-out-seqdir-sparse/tfidf-vectors Clusters In: examples/bin/work/cluster= s Out: examples/bin/work/reuters-kmeans Distance: org.apache.mahout.common.= distance.SquaredEuclideanDistanceMeasure >> 12/02/15 11:16:23 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterati= ons: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vecto= rs: {} >> 12/02/15 11:16:23 INFO kmeans.KMeansDriver: K-Means Iteration 1 >> 12/02/15 11:16:24 INFO input.FileInputFormat: Total input paths to proce= ss : 1 >> 12/02/15 11:16:24 INFO mapred.JobClient: Running job: job_201202131515_0= 126 >> 12/02/15 11:16:25 INFO mapred.JobClient: =C2=A0map 0% reduce 0% >> 12/02/15 11:16:38 INFO mapred.JobClient: Task Id : attempt_201202131515_= 0126_m_000000_0, Status : FAILED >> java.lang.IllegalStateException: No clusters found. Check your -c path. >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.mahout.clustering.kmeans.KMean= sMapper.setup(KMeansMapper.java:60) >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.mapreduce.Mapper.run(Ma= pper.java:142) >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.mapred.MapTask.runNewMa= pper(MapTask.java:621) >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.mapred.MapTask.run(MapT= ask.java:305) >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.mapred.Child.main(Child= .java:170) >> > Subject: Re: Mahout 0.5 java.lang.IllegalStateException: No clusters f= ound. Check your -c path. >> > From: suneel_marthi@yahoo.com >> > Date: Tue, 14 Feb 2012 21:50:22 -0500 >> > To: user@mahout.apache.org >> > >> > Did u specify the -cl option when executing kmeans? >> > >> > Sent from my iPhone >> > >> > On Feb 14, 2012, at 9:18 PM, Qiang Xu wrote: >> > >> > > >> > > I think there is nothing wrong with the path. >> > > >> > > Because the /user/root/examples/bin/work/clusters is generated by km= eans example. >> > > >> > > All my steps are: >> > > >> > > ./bin/mahout org.apache.lucene.benchmark.utils.ExtractReuters ./exam= ples/bin/work/reuters-sgm/ ./examples/bin/work/reuters-out/ >> > > >> > > ./bin/mahout seqdirectory -i ./examples/bin/work/reuters-out/ -o ./e= xamples/bin/work/reuters-out-seqdir -c UTF-8 -chunk 5 -ow >> > > >> > > ./bin/mahout seq2sparse -i ./examples/bin/work/reuters-out-seqdir/ -= o ./examples/bin/work/reuters-out-seqdir-sparse >> > > >> > > ./bin/mahout kmeans -i >> > > ./examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors/ -c >> > > ./examples/bin/work/clusters -o ./examples/bin/work/reuters-kmeans -= x 10 >> > > -k 20 -ow >> > > >> > > ./bin/mahout clusterdump -s examples/bin/work/reuters-kmeans/cluster= s-10 >> > > -d examples/bin/work/reuters-out-seqdir-sparse/dictionary.file-0 -dt >> > > sequencefile -b 100 -n 20 >> > > >> > > I have also tested with aboosolute path of hdfs as following: >> > > >> > > [root@qxutest mahout-distribution-0.5]# hadoop fs -ls /user/root/exa= mples/bin/work/ >> > > >> > > Found 4 items >> > > >> > > drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00 2012-02-14 20:55 /user/root/examples/bin/work/clusters >> > > >> > > drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00 2012-02-14 20:56 /user/root/examples/bin/work/reuters-kmeans >> > > >> > > drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00 2012-02-14 20:29 /user/root/examples/bin/work/reuters-out-seqdir >> > > >> > > drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00 2012-02-14 20:32 /user/root/examples/bin/work/reuters-out-seqdir-spars= e >> > > >> > > [root@qxutest mahout-distribution-0.5]# hadoop fs -ls /user/root/exa= mples/bin/work/clusters >> > > >> > > Found 1 items >> > > >> > > rw-rr- =C2=A0 2 root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0139 2012-= 02-14 20:55 /user/root/examples/bin/work/clusters/part-randomSeed >> > > >> > > [root@qxutest mahout-distribution-0.5]# >> > > ./bin/mahout kmeans -i >> > > /user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors= / -c >> > > =C2=A0/user/root/examples/bin/work/clusters -o >> > > /user/root/examples/bin/work/reuters-kmeans -x 10 =C2=A0-ow >> > > >> > > Running on hadoop, using HADOOP_HOME=3D/data/hadoop_cluster/hadoop-0= .20.2/ >> > > >> > > HADOOP_CONF_DIR=3D/data/hadoop_cluster/hadoop-0.20.2/conf/ >> > > >> > > 12/02/15 10:32:25 INFO common.AbstractJob: Command line arguments: >> > > {--clusters=3D/user/root/examples/bin/work/clusters, >> > > --convergenceDelta=3D0.5, >> > > --distanceMeasure=3Dorg.apache.mahout.common.distance.SquaredEuclide= anDistanceMeasure, >> > > --endPhase=3D2147483647, >> > > --input=3D/user/root/examples/bin/work/reuters-out-seqdir-sparse/tfi= df-vectors/, >> > > --maxIter=3D10, --method=3Dmapreduce, >> > > --output=3D/user/root/examples/bin/work/reuters-kmeans, --overwrite= =3Dnull, >> > > --startPhase=3D0, --tempDir=3Dtemp} >> > > >> > > 12/02/15 10:32:25 INFO common.HadoopUtil: Deleting /user/root/exampl= es/bin/work/reuters-kmeans >> > > >> > > 12/02/15 10:32:25 INFO kmeans.KMeansDriver: Input: >> > > /user/root/examples/bin/work/reuters-out-seqdir-sparse/tfidf-vectors >> > > Clusters In: /user/root/examples/bin/work/clusters Out: >> > > /user/root/examples/bin/work/reuters-kmeans Distance: >> > > org.apache.mahout.common.distance.SquaredEuclideanDistanceMeasure >> > > >> > > 12/02/15 10:32:25 INFO kmeans.KMeansDriver: convergence: 0.5 max >> > > Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritab= le >> > > Input Vectors: {} >> > > >> > > 12/02/15 10:32:25 INFO kmeans.KMeansDriver: K-Means Iteration 1 >> > > >> > > 12/02/15 10:32:26 INFO input.FileInputFormat: Total input paths to p= rocess : 1 >> > > >> > > 12/02/15 10:32:27 INFO mapred.JobClient: Running job: job_2012021315= 15_0123 >> > > >> > > 12/02/15 10:32:28 INFO mapred.JobClient: =C2=A0map 0% reduce 0% >> > > >> > > 12/02/15 10:32:38 INFO mapred.JobClient: Task Id : attempt_201202131= 515_0123_m_000000_0, Status : FAILED >> > > >> > > java.lang.IllegalStateException: No clusters found. Check your -c pa= th. >> > > >> > > =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.mahout.clustering.kmeans.KM= eansMapper.setup(KMeansMapper.java:60) >> > > >> > > =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapreduce.Mapper.run= (Mapper.java:142) >> > > >> > > =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapred.MapTask.runNe= wMapper(MapTask.java:621) >> > > >> > > =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapred.MapTask.run(M= apTask.java:305) >> > > >> > > =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapred.Child.main(Ch= ild.java:170) >> > > >> > >> From: xxqonline@hotmail.com >> > >> To: user@mahout.apache.org >> > >> Subject: RE: Mahout 0.5 java.lang.IllegalStateException: No cluster= s found. Check your -c path. >> > >> Date: Tue, 14 Feb 2012 23:47:53 +0800 >> > >> >> > >> >> > >> >> > >> I have checked 0.5 and 0.6 package, both of them have this problem.= Could you give me a work around or temp fixing?> From: xxqonline@hotmail.co= m >> > >>> To: user@mahout.apache.org >> > >>> Subject: Mahout 0.5 java.lang.IllegalStateException: No clusters f= ound. Check your -c path. >> > >>> Date: Tue, 14 Feb 2012 20:47:49 +0800 >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> Hello guys: =C2=A0 =C2=A0 =C2=A0 =C2=A0I am using Mahout 0.5,I fol= low the guide in https://cwiki.apache.org/MAHOUT/k-means-clustering.html to= run kmeans.But I got the following error.Mahout 0.5 java.lang.IllegalState= Exception: No clusters found. Check your -c path. =C2=A0It seems been fix i= n 0.4 https://issues.apache.org/jira/browse/MAHOUT-504?focusedCommentId=3D1= 3207675#comment-13207675But it is still in mahout 0.5Could some one give me= a work around way? Regards,skaterxu ./bin/mahout kmeans -i ./examples/bin/= work/reuters-out-seqdir-sparse/tfidf-vectors/ -c ./examples/bin/work/cluste= rs -o ./examples/bin/work/reuters-kmeans -x 10 =C2=A0-ow >> > >>> Running on hadoop, using HADOOP_HOME=3D/data/hadoop_cluster/hadoop= -0.20.2/ >> > >>> HADOOP_CONF_DIR=3D/data/hadoop_cluster/hadoop-0.20.2/conf/ >> > >>> 12/02/14 20:56:03 INFO common.AbstractJob: Command line arguments:= {--clusters=3D./examples/bin/work/clusters, --convergenceDelta=3D0.5, --di= stanceMeasure=3Dorg.apache.mahout.common.distance.SquaredEuclideanDistanceM= easure, --endPhase=3D2147483647, --input=3D./examples/bin/work/reuters-out-= seqdir-sparse/tfidf-vectors/, --maxIter=3D10, --method=3Dmapreduce, --outpu= t=3D./examples/bin/work/reuters-kmeans, --overwrite=3Dnull, --startPhase=3D= 0, --tempDir=3Dtemp} >> > >>> 12/02/14 20:56:03 INFO kmeans.KMeansDriver: Input: examples/bin/wo= rk/reuters-out-seqdir-sparse/tfidf-vectors Clusters In: examples/bin/work/c= lusters Out: examples/bin/work/reuters-kmeans Distance: org.apache.mahout.c= ommon.distance.SquaredEuclideanDistanceMeasure >> > >>> 12/02/14 20:56:03 INFO kmeans.KMeansDriver: convergence: 0.5 max I= terations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input= Vectors: {} >> > >>> 12/02/14 20:56:03 INFO kmeans.KMeansDriver: K-Means Iteration 1 >> > >>> 12/02/14 20:56:05 INFO input.FileInputFormat: Total input paths to= process : 1 >> > >>> 12/02/14 20:56:06 INFO mapred.JobClient: Running job: job_20120213= 1515_0122 >> > >>> 12/02/14 20:56:07 INFO mapred.JobClient: =C2=A0map 0% reduce 0% >> > >>> 12/02/14 20:56:16 INFO mapred.JobClient: Task Id : attempt_2012021= 31515_0122_m_000000_0, Status : FAILED >> > >>> java.lang.IllegalStateException: No clusters found. Check your -c = path. >> > >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.mahout.clustering.kmeans.= KMeansMapper.setup(KMeansMapper.java:60) >> > >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapreduce.Mapper.r= un(Mapper.java:142) >> > >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapred.MapTask.run= NewMapper(MapTask.java:621) >> > >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapred.MapTask.run= (MapTask.java:305) >> > >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.mapred.Child.main(= Child.java:170) >> > >>> It is really weired that cluster is gernerated >> > >>> [root@qxutest mahout-distribution-0.5]# hadoop fs -ls /user/root/e= xamples/bin/work/ >> > >>> Found 4 items >> > >>> drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A00 2012-02-14 20:55 /user/root/examples/bin/work/clusters >> > >>> drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A00 2012-02-14 20:56 /user/root/examples/bin/work/reuters-kmeans >> > >>> drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A00 2012-02-14 20:29 /user/root/examples/bin/work/reuters-out-seqdir >> > >>> drwxr-xr-x =C2=A0 - root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A00 2012-02-14 20:32 /user/root/examples/bin/work/reuters-out-seqdir-sp= arse >> > >>> [root@qxutest mahout-distribution-0.5]# hadoop fs -ls /user/root/e= xamples/bin/work/clusters >> > >>> Found 1 items >> > >>> rw-rr- =C2=A0 2 root supergroup =C2=A0 =C2=A0 =C2=A0 =C2=A0139 201= 2-02-14 20:55 /user/root/examples/bin/work/clusters/part-randomSeed >> > >> >> > > >> > --=20 Lance Norskog goksron@gmail.com