Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <230345297.1218200024469.JavaMail.jira@brutus>
Date: Fri, 8 Aug 2008 05:53:44 -0700 (PDT)
From: "Tom White (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3912) Improvements for NativeS3FileSystem
In-Reply-To: <90178646.1218051884385.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620925#action_12620925 ] 

Tom White commented on HADOOP-3912:
-----------------------------------

These look like good changes to have. It's hard to follow the source as so much has changed. It would be good to have a patch file, and some unit tests.

* 1. was fixed in HADOOP-3640.
* Rename should probably be done in HADOOP-3361 since this is the only part that needs the new JetS3t library - the next release is coming soon according to http://groups.google.com/group/jets3t-users/browse_thread/thread/ef4b7cc08c31443.

> Improvements for NativeS3FileSystem
> -----------------------------------
>
>                 Key: HADOOP-3912
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3912
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: fs/s3
>            Reporter: Albert Chern
>         Attachments: NativeS3FileSystem.java
>
>
> In the process of porting NativeS3FileSystem for use with Hadoop 12, I made the following changes and improvements which might be helpful (apologies if I should have opened separate issues, but I was lazy):
> 1. The single byte read() method of NativeS3InputStream incorrectly treats the return value of InputStream.read() as the number of bytes read, which is actually always 1.
> 2. It allows people to write files ending with the folder suffix.  I prevented this by doing a check in the create() method.
> 3. Similarly, it allows people to open directories for reading.  I prevented this by doing a check in the open() method.
> 4. If you write a file to a nonexistent directory tree, say /a/b/c/d/file, and then you delete/rename that file or one of its parent directories, the whole directory tree vanishes.  I fixed this by always creating the parent of a deleted/renamed file.
> 5. Recursive delete(), rename(), and getContentLength() can be sped up tremendously by working directly with S3 listings rather than working at the FileSystem level.  All sub-files/sub-directories should begin with the parent directory name as a prefix.
> 6. HADOOP-3506 is still relevant.
> I don't have patches since I created a new file, but I have attached my source if Tom wants to take a look at it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.