hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Telford (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-7269) S3 Native should allow customizable file meta-data (headers)
Date Thu, 12 May 2011 16:57:47 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Nicholas Telford updated HADOOP-7269:

    Attachment: HADOOP-7269-S3-metadata-001.diff

Attached is a patch that adds basic support for key/value metadata when creating files with
a FileSystem.

FileSystems that don't support metadata will simply discard any that is provided.

Keys are dependant on the FileSystem being used, in the case of s3native, they're used as
HTTP headers and should match the headers available here: http://docs.amazonwebservices.com/AmazonS3/latest/API/

I'm not sure how I feel about this approach. On the one hand, it's fairly abstract and flexible.
On the other, it makes the assertion that metadata is always a Map<String, String> which
may prove fairly restrictive. Still, I can't think of a better way to do it.

> S3 Native should allow customizable file meta-data (headers)
> ------------------------------------------------------------
>                 Key: HADOOP-7269
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7269
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>            Reporter: Nicholas Telford
>            Priority: Minor
>         Attachments: HADOOP-7269-S3-metadata-001.diff
> The S3 Native FileSystem currently writes all files with a set of default headers:
>  * Content-Type: binary/octet-stream
>  * Content-Length: <computed from object size>
>  * Content-MD5: <computed from object data>
> This is a good start, however many applications would benefit from the ability to customize
(for example) the Content-Type and Expires headers for the file. Ideally the implementation
should be abstract enough to customize all of the available S3 headers and provide a facility
for other FileSystems to specify optional file metadata.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message