hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王世森 <wangshi...@oraro.net>
Subject how to view the result of kmeans
Date Fri, 17 Sep 2010 06:42:31 GMT
hi,

 

I install the mahout and hadoop on SUSE Linux, and it works. 

I run the kmeans sample use command:

hadoop fs -put synthetic_control.data testdata

mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.Job

 

it run successfully here is the output files

drwxr-xr-x   - root supergroup          0 2010-09-17 10:59
/user/root/output/canopies

drwxr-xr-x   - root supergroup          0 2010-09-17 10:59
/user/root/output/canopies/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 10:59
/user/root/output/canopies/_logs/history

-rw-r--r--   1 root supergroup      17229 2010-09-17 10:59
/user/root/output/canopies/_logs/history/localhost_1284692116007_job_2010091
71055_0002_conf.xml

-rw-r--r--   1 root supergroup       9165 2010-09-17 10:59
/user/root/output/canopies/_logs/history/localhost_1284692116007_job_2010091
71055_0002_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5749 2010-09-17 10:59
/user/root/output/canopies/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:00
/user/root/output/clusters-0

drwxr-xr-x   - root supergroup          0 2010-09-17 10:59
/user/root/output/clusters-0/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 10:59
/user/root/output/clusters-0/_logs/history

-rw-r--r--   1 root supergroup      17479 2010-09-17 10:59
/user/root/output/clusters-0/_logs/history/localhost_1284692116007_job_20100
9171055_0003_conf.xml

-rw-r--r--   1 root supergroup       9283 2010-09-17 10:59
/user/root/output/clusters-0/_logs/history/localhost_1284692116007_job_20100
9171055_0003_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:00
/user/root/output/clusters-0/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:00
/user/root/output/clusters-1

drwxr-xr-x   - root supergroup          0 2010-09-17 11:00
/user/root/output/clusters-1/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:00
/user/root/output/clusters-1/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:00
/user/root/output/clusters-1/_logs/history/localhost_1284692116007_job_20100
9171055_0004_conf.xml

-rw-r--r--   1 root supergroup       9283 2010-09-17 11:00
/user/root/output/clusters-1/_logs/history/localhost_1284692116007_job_20100
9171055_0004_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:00
/user/root/output/clusters-1/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:01
/user/root/output/clusters-2

drwxr-xr-x   - root supergroup          0 2010-09-17 11:00
/user/root/output/clusters-2/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:00
/user/root/output/clusters-2/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:00
/user/root/output/clusters-2/_logs/history/localhost_1284692116007_job_20100
9171055_0005_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:00
/user/root/output/clusters-2/_logs/history/localhost_1284692116007_job_20100
9171055_0005_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:01
/user/root/output/clusters-2/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:01
/user/root/output/clusters-3

drwxr-xr-x   - root supergroup          0 2010-09-17 11:01
/user/root/output/clusters-3/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:01
/user/root/output/clusters-3/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:01
/user/root/output/clusters-3/_logs/history/localhost_1284692116007_job_20100
9171055_0006_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:01
/user/root/output/clusters-3/_logs/history/localhost_1284692116007_job_20100
9171055_0006_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:01
/user/root/output/clusters-3/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:02
/user/root/output/clusters-4

drwxr-xr-x   - root supergroup          0 2010-09-17 11:01
/user/root/output/clusters-4/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:01
/user/root/output/clusters-4/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:01
/user/root/output/clusters-4/_logs/history/localhost_1284692116007_job_20100
9171055_0007_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:01
/user/root/output/clusters-4/_logs/history/localhost_1284692116007_job_20100
9171055_0007_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:02
/user/root/output/clusters-4/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:02
/user/root/output/clusters-5

drwxr-xr-x   - root supergroup          0 2010-09-17 11:02
/user/root/output/clusters-5/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:02
/user/root/output/clusters-5/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:02
/user/root/output/clusters-5/_logs/history/localhost_1284692116007_job_20100
9171055_0008_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:02
/user/root/output/clusters-5/_logs/history/localhost_1284692116007_job_20100
9171055_0008_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:02
/user/root/output/clusters-5/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:03
/user/root/output/clusters-6

drwxr-xr-x   - root supergroup          0 2010-09-17 11:02
/user/root/output/clusters-6/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:02
/user/root/output/clusters-6/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:02
/user/root/output/clusters-6/_logs/history/localhost_1284692116007_job_20100
9171055_0009_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:02
/user/root/output/clusters-6/_logs/history/localhost_1284692116007_job_20100
9171055_0009_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:03
/user/root/output/clusters-6/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:03
/user/root/output/clusters-7

drwxr-xr-x   - root supergroup          0 2010-09-17 11:03
/user/root/output/clusters-7/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:03
/user/root/output/clusters-7/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:03
/user/root/output/clusters-7/_logs/history/localhost_1284692116007_job_20100
9171055_0010_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:03
/user/root/output/clusters-7/_logs/history/localhost_1284692116007_job_20100
9171055_0010_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:03
/user/root/output/clusters-7/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/clusters-8

drwxr-xr-x   - root supergroup          0 2010-09-17 11:03
/user/root/output/clusters-8/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:03
/user/root/output/clusters-8/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:03
/user/root/output/clusters-8/_logs/history/localhost_1284692116007_job_20100
9171055_0011_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:03
/user/root/output/clusters-8/_logs/history/localhost_1284692116007_job_20100
9171055_0011_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:03
/user/root/output/clusters-8/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/clusters-9

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/clusters-9/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/clusters-9/_logs/history

-rw-r--r--   1 root supergroup      17481 2010-09-17 11:04
/user/root/output/clusters-9/_logs/history/localhost_1284692116007_job_20100
9171055_0012_conf.xml

-rw-r--r--   1 root supergroup       9496 2010-09-17 11:04
/user/root/output/clusters-9/_logs/history/localhost_1284692116007_job_20100
9171055_0012_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup       5757 2010-09-17 11:04
/user/root/output/clusters-9/part-00000

drwxr-xr-x   - root supergroup          0 2010-09-17 10:59
/user/root/output/data

drwxr-xr-x   - root supergroup          0 2010-09-17 10:58
/user/root/output/data/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 10:58
/user/root/output/data/_logs/history

-rw-r--r--   1 root supergroup      16671 2010-09-17 10:58
/user/root/output/data/_logs/history/localhost_1284692116007_job_20100917105
5_0001_conf.xml

-rw-r--r--   1 root supergroup       6148 2010-09-17 10:58
/user/root/output/data/_logs/history/localhost_1284692116007_job_20100917105
5_0001_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup     240672 2010-09-17 10:59
/user/root/output/data/part-00000

-rw-r--r--   1 root supergroup     242288 2010-09-17 10:59
/user/root/output/data/part-00001

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/points

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/points/_logs

drwxr-xr-x   - root supergroup          0 2010-09-17 11:04
/user/root/output/points/_logs/history

-rw-r--r--   1 root supergroup      17109 2010-09-17 11:04
/user/root/output/points/_logs/history/localhost_1284692116007_job_201009171
055_0013_conf.xml

-rw-r--r--   1 root supergroup       6124 2010-09-17 11:04
/user/root/output/points/_logs/history/localhost_1284692116007_job_201009171
055_0013_root_mahout-examples-0.3.job

-rw-r--r--   1 root supergroup     515948 2010-09-17 11:04
/user/root/output/points/part-00000

-rw-r--r--   1 root supergroup     512632 2010-09-17 11:04
/user/root/output/points/part-00001

 

Then i download the output file to local use "hadoop fs -get output output"

but when i view the result that kmeans created used the command: mahout
clusterdump -s output/clusters-9 -p output/points

it throws a exception:

running on hadoop, using HADOOP_HOME=/usr/hadoop-0.20.2 and
HADOOP_CONF_DIR=/usr/hadoop-0.20.2/conf

10/09/17 14:33:29 ERROR driver.MahoutDriver: MahoutDriver failed with args:
[-s, output/clusters-9, -p, output/points, null]

File does not exist: /usr/mahout-0.3/output/points/part-00000

Exception in thread "main" java.io.FileNotFoundException: File does not
exist: /usr/mahout-0.3/output/points/part-00000

        at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSy
stem.java:457)

        at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:676)

        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)

        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)

        at
org.apache.mahout.utils.clustering.ClusterDumper.readPoints(ClusterDumper.ja
va:330)

        at
org.apache.mahout.utils.clustering.ClusterDumper.init(ClusterDumper.java:85)

        at
org.apache.mahout.utils.clustering.ClusterDumper.<init>(ClusterDumper.java:7
8)

        at
org.apache.mahout.utils.clustering.ClusterDumper.main(ClusterDumper.java:276
)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
)

        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver
.java:68)

        at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)

        at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:172)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
)

        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

 

so, how to view the result of kmeans?

 

Best Wishes

 

Wangss


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message