hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James P. White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-574) want FileSystem implementation for Amazon S3
Date Thu, 30 Nov 2006 22:15:25 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-574?page=comments#action_12454765 ] 
James P. White commented on HADOOP-574:

I was thinking that you would retain the parsing of the id and secret from the URI, but that
the secret would be found by id in the keystore/configuration file if omitted.

The same approach would apply to the id if omitted, as the configuration file could supply
a default id (fs.s3.awsAccessKeyId), which would then lead to the secret.

One way to do that would be to use the configuration file and put the id in the property name:


That ultimately would be compatible with the old schme if fs.s3.awsSecretAccessKey were the
default secret if not otherwise specified (effectively fs.s3.awsSecretAccessKey.*).

The only reservation I have with putting the secret in the configuration file is I don't like
using plain text for secrets.  An encrypted store would be nicer.  Supporting OpenSSH Agents
would be very nice too.

> want FileSystem implementation for Amazon S3
> --------------------------------------------
>                 Key: HADOOP-574
>                 URL: http://issues.apache.org/jira/browse/HADOOP-574
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 0.9.0
>            Reporter: Doug Cutting
>         Attachments: dependencies.zip, HADOOP-574-v2.patch, HADOOP-574.patch
> An S3-based Hadoop FileSystem would make a great addition to Hadoop.
> It would facillitate use of Hadoop on Amazon's EC2 computing grid, as discussed here:
> http://www.mail-archive.com/hadoop-user@lucene.apache.org/msg00318.html
> This is related to HADOOP-571, which would make Hadoop's FileSystem considerably easier
to extend.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message