mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Eastman <j...@windwardsolutions.com>
Subject Re: Mahout on Elastic MapReduce
Date Wed, 15 Apr 2009 18:19:24 GMT
None of the clustering implementations hard code the filesystem. The 
file names are constructed from the input and output filepath arguments.

Jeff

Grant Ingersoll wrote:
> I seem to recall this being something you have to set in your Hadoop 
> configuration.  Or, let me double check that we aren't hard-coding the 
> FS in our Job.
>
> -Grant
>
> On Apr 15, 2009, at 1:27 PM, Stephen Green wrote:
>
>> On Apr 14, 2009, at 6:54 PM, Stephen Green wrote:
>>> On Apr 14, 2009, at 5:17 PM, Grant Ingersoll wrote:
>>>
>>>> I would be concerned about the fact that EMR is using 0.18 and 
>>>> Mahout is on 0.19 (which of course raises another concern expressed 
>>>> by Owen O'Malley to me at ApacheCon: No one uses 0.19)
>>>
>>> Well, I did run Mahout locally on a 0.18.3 install, but that was 
>>> writing to and reading from HDFS.  I can build a custom 
>>> mahout-examples that has the 0.18.3 Hadoop jars (or perhaps no 
>>> hadoop jar at all...) I'm guessing if EMR is on 0.18.3 and it gets 
>>> popular, then you're going to have to deal with that problem.
>>
>>
>> More fun today.  I checked out the mahout-0.1 release and rebuilt 
>> mahout.  I took the mahout-examples job, removed the hadoop jar, and 
>> then tried to run the KMeans clustering against the synthetic control 
>> data.  This failed with the same exception that I was originally 
>> getting yesterday:
>>
>> java.lang.IllegalArgumentException: Wrong FS: s3n://mahout-output/, 
>> expected: hdfs://domU-12-31-38-01-C5-22.compute-1.internal:9000
>>        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:320)
>>        at 
>> org.apache.hadoop.dfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:84)

>>
>>        at 
>> org.apache.hadoop.dfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:140)

>>
>>        at 
>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:408)

>>
>>        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:667)
>>        at 
>> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.runJob(Job.java:77) 
>>
>>        at 
>> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:43) 
>>
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>>
>>        at 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

>>
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
>>        at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>        at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
>>
>> Steve
>> -- 
>> Stephen Green                      //   Stephen.Green@sun.com
>> Principal Investigator             \\   http://blogs.sun.com/searchguy
>> Aura Project                       //   Voice: +1 781-442-0926
>> Sun Microsystems Labs              \\   Fax:   +1 781-442-1692
>>
>>
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) 
> using Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
>


Mime
View raw message