hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-574) want FileSystem implementation for Amazon S3
Date Mon, 13 Nov 2006 19:22:40 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-574?page=comments#action_12449457 ] 
            
Doug Cutting commented on HADOOP-574:
-------------------------------------

> The other aspect I haven't put much thought into yet is locking.

I don't think locking is critical for for the first version.  MapReduce jobs shouldn't generally
require locking.  Longer term we should probably consider adding a locking service to Hadoop
directly, rather than relying on one in the filesystem implementation.

If no one else steps up to the plate in the next day or so then I'll start implementing HADOOP-571
(uri-based FileSystem Paths).  I'd like to get that into this month's release if possible.

I also added a related thread to Amazon's EC2 forum.

http://developer.amazonwebservices.com/connect/thread.jspa?threadID=12677

When you get something working, please attach it as a patch to this issue.  Thanks!


> want FileSystem implementation for Amazon S3
> --------------------------------------------
>
>                 Key: HADOOP-574
>                 URL: http://issues.apache.org/jira/browse/HADOOP-574
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Doug Cutting
>
> An S3-based Hadoop FileSystem would make a great addition to Hadoop.
> It would facillitate use of Hadoop on Amazon's EC2 computing grid, as discussed here:
> http://www.mail-archive.com/hadoop-user@lucene.apache.org/msg00318.html
> This is related to HADOOP-571, which would make Hadoop's FileSystem considerably easier
to extend.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message