hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-574) want FileSystem implementation for Amazon S3
Date Tue, 14 Nov 2006 22:13:40 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-574?page=all ]

Tom White updated HADOOP-574:
-----------------------------

               Status: Patch Available  (was: Open)
    Affects Version/s: 0.9.0

Please find attached a patch for an S3 filesystem for review and discussion. It is not yet
complete so I am not expecting it to be committed.
There are a number of TODO comments in the code that list the remaining work items. These
mainly concern corner cases or tidying up work - all the basic file system operations work.

The code has been exercised by unit tests. To get these to run you need to fill in S3 access
codes in the test hadoop-site.xml. (This raises an interesting question of how to have these
run as automated tests.)

I have not tried running Hadoop (or something like DfsShell) with this filesystem. We probably
need to wait until HADOOP-571 is done.

Looking forward to your feedback!

> want FileSystem implementation for Amazon S3
> --------------------------------------------
>
>                 Key: HADOOP-574
>                 URL: http://issues.apache.org/jira/browse/HADOOP-574
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 0.9.0
>            Reporter: Doug Cutting
>
> An S3-based Hadoop FileSystem would make a great addition to Hadoop.
> It would facillitate use of Hadoop on Amazon's EC2 computing grid, as discussed here:
> http://www.mail-archive.com/hadoop-user@lucene.apache.org/msg00318.html
> This is related to HADOOP-571, which would make Hadoop's FileSystem considerably easier
to extend.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message