Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\))
Subject: Re: example usage of s3 file system
From: Chris Collins <chris_j_collins@yahoo.com>
In-Reply-To: 
 <CAOcnVr1GbDcWJ5RM=qqtpHxYf2N67t5XsS2eFFudyTe=MgxzVQ@mail.gmail.com>
Date: Wed, 29 Aug 2012 08:44:08 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <66658CFB-950B-48C2-9648-A8C9FECAA90D@yahoo.com>
References: <FD7035C6-E02E-4C90-9890-16DC179C31FD@yahoo.com>
 <CAKH910-3xVjnEE7PJxvKGe5pj3xWZ6oxNV0xK2im6+KpkVM4qw@mail.gmail.com>
 <257DBE47-8581-4FCE-886B-F010DBB45BCB@yahoo.com>
 <CAOcnVr1GbDcWJ5RM=qqtpHxYf2N67t5XsS2eFFudyTe=MgxzVQ@mail.gmail.com>
To: user@hadoop.apache.org

No problem.  Didnt get anything from the issues email alias.  Seems that =
also the url that Haavard pointed to probably also needs to say what =
jets3t jar file to include.  People probably have a tendency to want to =
use the latest version (which is 9) where you seem to have to use 7.x to =
get hadoop not to give a method invocation exception.  I am guessing it =
cant be that important though cause for the life of me I cant understand =
how it ever worked for native (well at least doing a mkdir).

C
On Aug 29, 2012, at 5:48 AM, Harsh J <harsh@cloudera.com> wrote:

> Many thanks for taking this upstream with a fix Chris!
>=20
> On Wed, Aug 29, 2012 at 12:51 PM, Chris Collins
> <chris_j_collins@yahoo.com> wrote:
>> Thanks Haavard, I am aware of that page but I am not sure why you are =
pointing me to it.  This really looks like a bug where =
Jets3tNativeFileSystemStore is parsing a response from jets3t.  its =
looking for ResponseCode=3D404 but actually getting ResponseCode: 404.  =
I dont see how it ever worked looking back through the versions of =
release code.
>>=20
>> I took a copy of the s3native package and made a fix and seems to get =
around the issue.
>>=20
>> I have reported it to the issues email alias, I will see if there is =
actually any interest in this problem.
>>=20
>> Cheers
>>=20
>> C
>> On Aug 29, 2012, at 12:11 AM, H=E5vard Wahl Kongsg=E5rd =
<haavard.kongsgaard@gmail.com> wrote:
>>=20
>>> see also
>>>=20
>>> http://wiki.apache.org/hadoop/AmazonS3
>>>=20
>>> On Tue, Aug 28, 2012 at 9:14 AM, Chris Collins
>>> <chris_j_collins@yahoo.com> wrote:
>>>> Hi I am trying to use the Hadoop filesystem abstraction with S3 but =
in my tinkering I am not having a great deal of success.  I am =
particularly interested in the ability to mimic a directory structure =
(since s3 native doesnt do it).
>>>>=20
>>>> Can anyone point me to some good example usage of Hadoop FileSystem =
with s3?
>>>>=20
>>>> I created a few directories using transit and AWS S3 console for =
test.  Doing a liststatus of the bucket returns a FileStatus object of =
the directory created but if I try to do a liststatus of that path I am =
getting a 404:
>>>>=20
>>>> org.apache.hadoop.fs.s3.S3Exception: =
org.jets3t.service.S3ServiceException: Request Error. HEAD '/aaaa' on =
Host ....
>>>>=20
>>>> Probably not the best list to look for help, any clues appreciated.
>>>>=20
>>>> C
>>>=20
>>>=20
>>>=20
>>> --
>>> H=E5vard Wahl Kongsg=E5rd
>>> Faculty of Medicine &
>>> Department of Mathematical Sciences
>>> NTNU
>>>=20
>>> http://havard.security-review.net/
>>=20
>=20
>=20
>=20
> --=20
> Harsh J