mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Hitchcock <adpow...@gmail.com>
Subject Re: Mahout on Elastic MapReduce
Date Thu, 05 Aug 2010 21:00:07 GMT
I think you are missing a slash in the URI:

s3:/testingbucket-12345/tmp/prePartialMultiply2

What happens if you try:

s3://testingbucket-12345/tmp/prePartialMultiply2

Andrew

On Thu, Aug 5, 2010 at 5:51 AM, Sebastian Schelter
<ssc.open@googlemail.com> wrote:
> Hi,
>
> I'm currently evaluating ItemSimilarityJob and RecommenderJob on Elastic
> MapReduce, it seems we have some small problems with S3, mostly due to
> the fact that we need to use Filesystem.get(path.toUri(), conf) instead
> of Filesystem.get(conf) in the code. I will create a patch for that the
> next days.
>
> I'm writing this mail because I encountered another problem I currently
> can't solve. RecommenderJob is emulating MultipleInputs (which is
> currently missing in Hadoop 0.20 AFAIK) by reading data from a combined
> path that is built like that:
>
>    new Path(prePartialMultiplyPath1 + "," + prePartialMultiplyPath2)
>
> My Job always fails with this exception here:
>
>    java.lang.IllegalArgumentException: Invalid hostname in URI
> s3:/testingbucket-12345/tmp/prePartialMultiply2
>
> Any ideas how to fix this?
>
> Thanks,
> Sebastian
>
> (Wrong email in the last mail, sry)
>
> ###
>
> Stacktrace (line numbers might not correspond with the latest version
> from HEAD):
>
>
> Exception in thread "main" java.lang.IllegalArgumentException: Invalid
> hostname in URI s3:/testingbucket-12345/tmp/prePartialMultiply2
>    at
> org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:41)
>    at
> org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.initialize(Jets3tNativeFileSystemStore.java:53)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>    at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>    at org.apache.hadoop.fs.s3native.$Proxy2.initialize(Unknown Source)
>    at
> org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:278)
>    at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1418)
>    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>    at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1443)
>    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1431)
>    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
>    at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:203)
>    at
> org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)
>    at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:241)
>    at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:908)
>    at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:802)
>    at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
>    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
>    at
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:241)
>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>    at
> org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:286)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>    at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:597)
>    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
>

Mime
View raw message