hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Hanna <jeremy.hanna1...@gmail.com>
Subject Re: IllegalArgumentException when doing fs.open with an s3n prefix path
Date Thu, 10 Feb 2011 02:19:37 GMT
Wow, sorry, that was just my sad excuse.  Thanks again.

On Feb 9, 2011, at 6:53 PM, Andrew Hitchcock wrote:

> Ah, nice catch. I'll go fix that message now :)
> 
> On Wed, Feb 9, 2011 at 4:50 PM, Jeremy Hanna <jeremy.hanna1234@gmail.com> wrote:
>> Bah - you're right.  I don't know why I thought the real error was obscured, besides
being distracted by "you should of" should be "you should have".
>> 
>> Thanks and apologies...
>> 
>> Jeremy
>> 
>> On Feb 9, 2011, at 6:10 PM, Andrew Hitchcock wrote:
>> 
>>> "This file system object (hdfs://ip-10-114-89-36.ec2.internal:9000)
>>> does not support access to the request path
>>> 's3n://backlog.dev/1296648900000/32763897924550656' You possibly
>>> called FileSystem.get(conf) when you should of called
>>> FileSystem.get(uri, conf) to obtain a file system supporting your
>>> path."
>>> 
>>> That explains the error. You should always use the two parameter get
>>> method when requesting FileSystem objects.
>>> 
>>> Also, Elastic MapReduce is based on the Hadoop 0.20 branch. It has all
>>> the patches from Hadoop 0.20.2 plus some additional ones from that
>>> branch and other places.
>>> 
>>> Andrew
>>> 
>>> On Wed, Feb 9, 2011 at 3:23 PM, Jeremy Hanna <jeremy.hanna1234@gmail.com>
wrote:
>>>> Anyone know why I would be getting an error doing a filesystem.open on a
file with a s3n prefix?
>>>> 
>>>> for the input path "s3n://backlog.dev/1296648900000/" - I get the following
stacktrace:
>>>> 
>>>> java.lang.IllegalArgumentException: This file system object (hdfs://ip-10-114-89-36.ec2.internal:9000)
does not support access to the request path 's3n://backlog.dev/1296648900000/32763897924550656'
You possibly called FileSystem.get(conf) when you should of called FileSystem.get(uri, conf)
to obtain a file system supporting your path.
>>>>        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:351)
>>>>        at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:99)
>>>>        at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:155)
>>>>        at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:178)
>>>>        at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:396)
>>>>        at analytics.hadoop.socialdata.RawSignalFileInputFormat$MultiFileLineRecordReader.<init>(RawSignalFileInputFormat.java:53)
>>>>        at analytics.hadoop.socialdata.RawSignalFileInputFormat.getRecordReader(RawSignalFileInputFormat.java:22)
>>>>        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:343)
>>>>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:312)
>>>>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>>> 
>>>> Incidentally, I'm using on elastic mapreduce with hadoop version 0.20 (which
I assume is the latest 0.20 version).
>> 
>> 


Mime
View raw message