hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: example usage of s3 file system
Date Wed, 29 Aug 2012 12:48:30 GMT
Many thanks for taking this upstream with a fix Chris!

On Wed, Aug 29, 2012 at 12:51 PM, Chris Collins
<chris_j_collins@yahoo.com> wrote:
> Thanks Haavard, I am aware of that page but I am not sure why you are pointing me to
it.  This really looks like a bug where Jets3tNativeFileSystemStore is parsing a response
from jets3t.  its looking for ResponseCode=404 but actually getting ResponseCode: 404.  I
dont see how it ever worked looking back through the versions of release code.
> I took a copy of the s3native package and made a fix and seems to get around the issue.
> I have reported it to the issues email alias, I will see if there is actually any interest
in this problem.
> Cheers
> C
> On Aug 29, 2012, at 12:11 AM, Håvard Wahl Kongsgård <haavard.kongsgaard@gmail.com>
>> see also
>> http://wiki.apache.org/hadoop/AmazonS3
>> On Tue, Aug 28, 2012 at 9:14 AM, Chris Collins
>> <chris_j_collins@yahoo.com> wrote:
>>> Hi I am trying to use the Hadoop filesystem abstraction with S3 but in my tinkering
I am not having a great deal of success.  I am particularly interested in the ability to mimic
a directory structure (since s3 native doesnt do it).
>>> Can anyone point me to some good example usage of Hadoop FileSystem with s3?
>>> I created a few directories using transit and AWS S3 console for test.  Doing
a liststatus of the bucket returns a FileStatus object of the directory created but if I try
to do a liststatus of that path I am getting a 404:
>>> org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException: Request
Error. HEAD '/aaaa' on Host ....
>>> Probably not the best list to look for help, any clues appreciated.
>>> C
>> --
>> Håvard Wahl Kongsgård
>> Faculty of Medicine &
>> Department of Mathematical Sciences
>> http://havard.security-review.net/

Harsh J

View raw message